Having Problems Scraping with Proxies

sevenfigurez

Junior Member
Joined
Nov 5, 2009
Messages
184
Reaction score
46
I built a script using curl that scrapes google no problem. Once I implement the proxies, I immediately get hit with "Sorry, but your computer or network may be sending automated queries". Makes no sense...I can use VPN, change proxies, and re run script, but when I try to use proxies to run it it gets blocked.

What can I be missing?
 
Are you saving and reusing your cookies ?
Are you passing back the verifications calls from page to page ?

There are several factors that identify your calls as a BOT to ban you really fast.

Also make sure you have a clean list of proxies most public ones are already blacklisted and will present the message you mentioned
 
I do the same thing.. Some things you need to make sure of.

Make sure your using anonymous proxies.
stay away from public proxies that have been abused and not passing google traffic by investing in a good proxy list, Exclusive proxy list, Shared proxies or Private proxies.

What I do in my script is first test the proxies to see if they are working and passing google, Then rotate a proxy for each search. Seems to work great.
 
Back
Top