1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Approaches To Scrape Amazon for keyword research?

Discussion in 'Programming' started by PortScan, Jun 6, 2017.

  1. PortScan

    PortScan Newbie

    Joined:
    Nov 26, 2015
    Messages:
    19
    Likes Received:
    2
    Amazon has been trying harder and harder to stop scrapers (obviously). But there are many businesses that have made a business from scraping amazon.

    IE. JungleScout, CamelCamelCamle, SellerLabs..

    Is there a systematic crawler I can use, or do I have to do it case by case for each category. The Api is only useful after you have a asin, but you need to somehow get that huge dump of ASINS and keep up to date with new products.

    Currently, my strategy is just to make custom rules for each category until I get the paginated results and just scrape from there using proxies.

    I will be costly probably $200-300 a month for the proxies needed.

    I think search pages can be scraped with HTTP, but reviews will need PhantomJs.

    Anyone attempt this? What was the cost?
     
  2. Sergio-341

    Sergio-341 Newbie

    Joined:
    Jun 21, 2017
    Messages:
    31
    Likes Received:
    6
    Gender:
    Male
    Occupation:
    Seller
    Location:
    USA
    Home Page:
    why you want to scrape them... any specific use case?
     
  3. akio_daichi

    akio_daichi Newbie

    Joined:
    Jul 4, 2017
    Messages:
    2
    Likes Received:
    0
    Gender:
    Male
    What actually you want to scrape? As far as proxies are concerned, why you don't setup tor? It'll slow down the scrapping, but you'll have to only spend 5$ for a sever.