What is the best and cheapest solution with proxy or VPN to scrape Google?

What is the best and cheapest solution with proxy or VPN to scrape Google? I don't want scraping services with API because it's too expensive for this volume

I need to scrape 1 million serps per day.

I'm currently on mobile proxy, it works pretty well but I would like to test other solutions to do more volume.

Thanks for your recommendations

AWS+Puppeteer-stealth or fakebrowser

You can either code a script to get instances with new IPS all the time

or use AWS Lamba cloud functions, with default quota you can run 500 lambda functions at once per region(there is about 16 regions), each with a different IP, you can probably do 5 or 10 requests before they ask for a captcha, after that you have to trigger a cold boot on the functions, and that'll get you new IPS after 10 seconds

AWS Lambda base pricing is a bit more than 0.2$ per million requests(+ a bit more money depending on how long it takes you to scrape)

if you have AWS credits, it can be literally free to makes tens of millions of requests

I'm going to test but from memory I was getting a captcha directly on the first call. This discouraged me. But I will try again

42rankpbn said:

I'm going to test but from memory I was getting a captcha directly on the first call. This discouraged me. But I will try again

Click to expand...

Yeah I forgot to mention a few things.
The browser needs to be headfull, that means you need to run it on a server with a graphical desktop(like a regular windows or X server on linux), not hidden windows.
using the puppeteer-stealth plugin or fakebrowser is a must.
Also maybe you need to type the keywords from google.com frontpage, with a slight delay(there is example code for that on Fakebrowser github), it's probably not mandatory, but you'll be able to make more requests before getting a captcha