Is there a cheaper way to scrape Google than apify?

jamie3000

Elite Member
Executive VIP
Jr. VIP
Joined
Jun 30, 2014
Messages
13,738
Reaction score
5,214
Hi all,

I want to connect with someone scraping Google at scale.

I basically need LinkedIn data but it’s easier to pull it from Google results than LinkedIn.

So I need to know if there is a cheaper way to scrape Google than apify?

What’s everyone doing?

Thanks
Jamie
 
Yeah DataforSEO will be a lot cheaper than Apify, though their results especially in non-US geos can be wrong/look heavily cached. For just scraping Linkedin results en masse it shouldn't affect things though. I personally use A-Parser to scrape Google, worth looking at if you have a decent proxy set. They have a sales thread in the marketplace that I was only recently made aware of.
 
Anyone performing this activity on a large scale does not scrape diretly because proxy services cost a lot of money. The combination of SERP api or small targeted scrapers is usually more economical
 
If you're scraping Google at scale, rotating residential proxies can be a more affordable option than Apify. Pairing them with tools like Selenium or Playwright works well for many setups. Just keep in mind that you will still need to deal with CAPTCHAs, rate limits, and occasional changes to Google's search results.
 
Yeah DataforSEO will be a lot cheaper than Apify, though their results especially in non-US geos can be wrong/look heavily cached. For just scraping Linkedin results en masse it shouldn't affect things though. I personally use A-Parser to scrape Google, worth looking at if you have a decent proxy set. They have a sales thread in the marketplace that I was only recently made aware of.
I think DataforSEO is x5 its normal price because I'm using site:linkedin.com, but I'll do some tests and check. Thanks man.
 
Your cheapest option will be to plug in your own mobile phone, control it via ADB (assuming it's android) and automate your own mobile browser. Perfect fingerprint, and as long as you don't abuse it, you won't get rate limited.

I need more scaling and also I can’t even put my phone down lol. Although I do have 10 other spare android phones. I guess I’d need a sim only contract for each.

I think I’d hit a lot of captchas doing site: but I can pay for the solving services.

The good thing about apify is that it doesn’t charge more for site:
 
Using a full browser for every Google search sounds expensive for this.

I’d try a simple SERP parser first and only use Playwright when the normal request fails. Probably a lot cheaper than sending everything through Apify
 
Using a full browser for every Google search sounds expensive for this.

I’d try a simple SERP parser first and only use Playwright when the normal request fails. Probably a lot cheaper than sending everything through Apify

Because I need to do a lot of site:LinkedIn.com I might check out how the bing index compares for LinkedIn. I know bing is easier to scrape but their index is not as good as Google’s. But maybe LinkedIn will be an exception because it’s owned by Microsoft…
 
Back
Top