Scraping Google in 2022

dannyduberstein

dannyduberstein

Regular Member
Joined
Nov 1, 2011
Messages
214
Reaction score
117
I've been in IM for awhile and have scraped google during most of that time. However, the last few years I took a break and haven't scraped google at all. Before you could easily pay for any backconnect proxy service and could scrape google 24/7 without any issues. Now it seems things have changed a bit and it has gotten much harder. I'm aware there are serp api's that easily allow you to scrape google but i'd rather not go with these as i'll be scraping quite a bit. Are you guys using capctcha solutions such as Xevil with your backconnect proxies now, to better scrape google? Are there any tricks to get better results while scraping google? Any recommended proxy services? How is everyone dealing with the increased difficulty of scraping google?
 
Last edited:
Advertise on BHW
I just use a lot of different proxies and rotate them as I am scraping. I only use a proxy for a single query before I rotate it, and I have long random wait times between each query. I don't use any captcha solver: if I get a captcha, I just put that proxy on hold for a while and try that query with a different proxy.

Also OP, depending on what you are scraping for, it might be better to scrape from bing instead as they ban a lot less
 
i ran google scraping services from 2009-2020, put simple
google became very aggressive at blocking ip ranges as
google scraping took of. as the years went by google hit
so many ip block it wasnt worth the time to do it anymore.
though a few services still do
 
I scrape a lot and Google is also there, scraping is easy if you have good quality proxies.
Residential/4G/5G are the best ones for that.
In addition to that having captcha solving solution is a bonus.
 
dannyduberstein said:
Any recommended proxy services?
Click to expand...
You are gonna have to find that out yourself eventually by testing the service against your use case. There are many proxy providers that offer trials as well so that is recommended. You can find'em in the "Proxies for Sale" section.
 
I think they have become too aggressive, that captcha speedbump always appear even when doing it manually.
 
actually, google is the easiest website to scrape anonymous-proxies residential with a simple script will get the job done.
 
A decent proxy will allow 1 search / minute without giving you any captchas.
That's about 1,400 searches/day/proxy.

20 dedicated proxies should cost about $25/month and can give you more than 30 000 searches/day in total, which should be enough.
 
MKSKS said:
I scrape a lot and Google is also there, scraping is easy if you have good quality proxies.
Residential/4G/5G are the best ones for that.
In addition to that having captcha solving solution is a bonus.
Click to expand...
What captcha solution do you use?
 
Oxylabs and luminati have serps and scraping tools where you only pay per successful request.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top