1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Amazon scraper + ghosted Ip?

Discussion in 'BlackHat Lounge' started by Euphorix, Oct 19, 2015.

  1. Euphorix

    Euphorix Supreme Member

    Feb 20, 2015
    Likes Received:
    Mountain View, CA
    Hey there,

    General idea: Have an ebay to amazon scraper that finds Amazon items and pulls ASIN #s into a text file.

    Everything started out fine, i ran the bot at about 200 items and it pulls a bunch of ASIN's... but once I bumped the scraper up to 1k items it came up with an error and now will not pull ANY ASIN's. It runs through all items within a minute without pulling anything.. whereas before it would take 5 to 10 minutes

    Is it possible that amazon ghosted my ip? If so, why and what is the best course of action to take

    Also, I would really love to be able to use this thing on a mass scale. Is there any way to give this bot the ability to scrape thousands of items (on both ebay and amazon) without running into a problem?

    Out of my realm of understanding so any help/advice is seriously appreciated

    • Thanks Thanks x 1
  2. RightFootFanatic

    RightFootFanatic Regular Member

    May 31, 2015
    Likes Received:
    How are you scraping atm?

    By using the official APIs it should be easy to get what you want.

    Afaik Ebay has a max call limit per 24h, which starts at 5000 calls.
    That limit could be raised if you get your app approved (afaik).

    Amazon had some limits that were only x calls/second (60seconds). It's somewhere in their tos.
    • Thanks Thanks x 2