1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Anyone tried scraping eBay?

Discussion in 'General Programming Chat' started by seeplusplus, Jul 12, 2013.

  1. seeplusplus

    seeplusplus Power Member

    Joined:
    Aug 18, 2008
    Messages:
    511
    Likes Received:
    163
    I've been looking at getting into scraping for a while, and have found that Python has a lot of good functions available.

    Thing is though, I want to scrape data from eBay mainly, and was wondering if they have many/any methods in place which makes it very hard to do.

    If anyone has any advice or info they could share I'd really appreciate it.

    Thanks :)
     
  2. divok

    divok Senior Member

    Joined:
    Jul 21, 2010
    Messages:
    1,015
    Likes Received:
    634
    Location:
    http://twitter.com/divok
    install scrapy (Python) , read it's docs and you could be scraping ebay in a day .
     
    • Thanks Thanks x 1
  3. jamb0ss

    jamb0ss Junior Member

    Joined:
    Feb 9, 2012
    Messages:
    125
    Likes Received:
    45
    Occupation:
    Bots programming
    i can help you, pm me
     
    • Thanks Thanks x 1
  4. Gophering

    Gophering Junior Member Premium Member

    Joined:
    Mar 21, 2013
    Messages:
    115
    Likes Received:
    279
    Occupation:
    Unemployed
    Location:
    EU
    I did this a couple of years ago for a project, don't remember it being overly difficult. However, things might have changed of course. Others have already suggested some good scraping libraries for python. You might also want to look into phantomjs and casperjs (python bindings are available) if you need that extra authenticity that a headless browser can provide.
     
    • Thanks Thanks x 1
  5. alphaDomega

    alphaDomega Registered Member

    Joined:
    Sep 28, 2013
    Messages:
    66
    Likes Received:
    8
    Occupation:
    full-Time Dreamer
    Location:
    Uk
    In case you have not got to grips with scraping ebay then scrapy is a great framework there again using python it's very easy to make your own spider\scraper