I'm using squidproxies, but they don't let me scrape anymore. WTF.
I need to scrape some stuff off Google. The proxies must allow scraping.
Anybody know where I can get some scraping proxies?
the problem is not with the proxy service itself, it is how you are using them. if you blast thousands of queries at google, they will blacklist that ip temporarily. keep repeating this and youll get blacklisted for longer/permanently.
the point is, any ip number, from any provider, will get banned. you need to either scrape less aggressively, spread over a larger number of proxies, or ind an option to rotate through fresh addresses when they get blacklisted.