1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Legality of running scrappers

Discussion in 'Black Hat SEO' started by ArtOfVaw, Feb 13, 2017.

  1. ArtOfVaw

    ArtOfVaw Junior Member

    Nov 23, 2016
    Likes Received:
    I've build a scrapper and meant to run it on a specific well-known e-commerce site, think it as ebay/amazon/etc.

    The scrapper itself is functional more or less, but now I have concerns regarding the following points:

    • Is there any problem running it from my personal machine? or should I use a VPS? w/ or w/o proxies? The concern here is, is it ok to be ME who is running the scrapper, or is it not?
    • There is virtually no delay time between each page refresh and the such. Should I implement a delay mechanism or leave it be? The only delay here is when the bot is extracting data, which can be done in an instance.
    • What about the user agent? Doing a little quick research revealed that it's best to keep it a a browser.

    Pretty new to this scrapping business myself, so it's also easy for any newbie to wreck havoc without noticing.