Any new tips for scraping Google?

jamie3000

Elite Member
Executive VIP
Jr. VIP
Joined
Jun 30, 2014
Messages
13,741
Reaction score
5,218
I haven't scraped Google on a large / reliable scale for over a year now and just wanted to know if anything had changed?

Who else is doing this and what does your setup look like?

How's everyone handling recaptcha V2? Xevil, anti captcha or something custom?

Any input appreciated.

Thanks :-)
 
Handling captcha is just not worth it in this particular case, just use poxy rotation.

One neat trick that i could recommend is to push the results number per page to 100. It will save you a shittone ammount of calls.
 
Handling captcha is just not worth it in this particular case, just use poxy rotation.

One neat trick that i could recommend is to push the results number per page to 100. It will save you a shittone ammount of calls.

Thanks doing that already. Using anti captcha at the moment because I can't find a proxy pool that hasn't been utterly raped by everyone else. I think @loopline alluded to the fact most proxy pools seemed to get burnt out eventually.
 
The most consistent google scrape for me for some years has been hrefer for the shotgun approach, for accuracy I would go scrapebox reliant on decent proxy.
 
Thanks doing that already. Using anti captcha at the moment because I can't find a proxy pool that hasn't been utterly raped by everyone else. I think @loopline alluded to the fact most proxy pools seemed to get burnt out eventually.

Most rotating proxy services seem to be dead for scraping Google.
 
Thanks doing that already. Using anti captcha at the moment because I can't find a proxy pool that hasn't been utterly raped by everyone else. I think @loopline alluded to the fact most proxy pools seemed to get burnt out eventually.
Its not really a case of burnt out, although google does perma ban ips, but Id say thats the exception not the rule, but the rules change every day with google so...

Its about speed. If the total user base of the ip pool is going faster then google allows, the ips get mostly banned. If all the users cancel except 1, then in 48 hours those ips are unbanned and the 1 users has a hay day.

the issue is most pools are sold to the point that the user base is so great it constantly goes so fast most ips are banned by google.

If you find a service that is not sold to that point, then it will be shortly. Storm proxies was awesome for a long time, but then google change the rules and made it tighter, so its the same old same old, its a fight with google and ips.

But just because a pool doesn't work today, doesn't mean a bunch of people won't cancel and it will be fine next month. Or maybe it won't, maybe more people will sign up and it will be worse.

Do you need to scrape google?

Can you develop a footprint that gives you good results with bing?

Better questions get you better answers and better answers lead you to solutions that may be able to avert the whole issue.

my 2 cents.
 
Aparser is very good scrapper and will save you a lot of time if you want to write own scripts
 
Hey, @azypka thanks for the suggestion.From many days I was searching for this type of Scraper.And yes it working good than otgers.
 
ScrapeBox + Rotating Proxies works well.

My setup is VPS + ScrapeBox + Rotating Proxies Package.

You can check @loopline's YT Channel for all the guides on ScrapeBox.
 
ScrapeBox + Rotating Proxies works well.

My setup is VPS + ScrapeBox + Rotating Proxies Package.

You can check @loopline's YT Channel for all the guides on ScrapeBox.

Here is a video on how to setup the rotating proxies with scrapebox and the rest of the guides are on this same channel, nearly 150 videos

 
Back
Top