I've build a scrapper and meant to run it on a specific well-known e-commerce site, think it as ebay/amazon/etc. The scrapper itself is functional more or less, but now I have concerns regarding the following points: Is there any problem running it from my personal machine? or should I use a VPS? w/ or w/o proxies? The concern here is, is it ok to be ME who is running the scrapper, or is it not? There is virtually no delay time between each page refresh and the such. Should I implement a delay mechanism or leave it be? The only delay here is when the bot is extracting data, which can be done in an instance. What about the user agent? Doing a little quick research revealed that it's best to keep it a a browser. Pretty new to this scrapping business myself, so it's also easy for any newbie to wreck havoc without noticing.