@scritty - yeah, some people are eager and don't understand the IP block system. they set wait times too low, way too low. Been scraping google with semi-dedicated proxies for the past few days. 22 proxies set at 20-75 second wait between requests. I'm asked to enter captcha when I start because the other users of the proxies are noobs and topped them but after I enter captcha they run all day long witout any issues.
Some simple math for the mentally impaired:
* 20-75 second delay per IP per request = 47.5 seconds average delay per IP per request.
* 75 requests/hour on average
* 1800 requests/day
* 20 proxies ($20/mo) => 36000 requests/day
* 54000 requests/month/proxy
* 1080000 requests/month with 20 proxies.
I think it's pretty decent. You could probably hit them harder but I prefer to stay safe than get blocked. For PR checking you can certainly use more aggressive wait times (10-20 sec or even less) but for that I use direct connection and just reconnect my router when the IP gets blocked.
I'm using this heavily with RankTracker and also sometimes with ScrapeBox. With ScrapeBox I prefer to leech 3-400 public proxies though because they get the job done and don't care if they die anyway.
I'm using semi-dedicated proxies from xxxxxxx and very happy with them - good price, great speed, good C-class diversity, they have good support (impressive reply time to support requests compared to others) and keep them running smooth all the time.
Scritty, what wait times you use between requests?