Scrape Google with python

Toublick

Newbie
Joined
Jul 12, 2020
Messages
13
Reaction score
4
Hello,
I'm scraping Google with a python script (with Requests library).
The URL of Google show 100 search results and I open differents URL with different keywords.
I run the python script on my Desktop.
If I do a Google request every 40 seconde, Google show me the captcha.
If I do a Google request every 50 or 55 seconde, no problem, my script can run several days without been blocked.

BUT (and that's why I create this thread on this forum) when I use the same IP BUT different devices (an Android smartphone and my computer) that share the same IP, I can run the 2 scripts in the same time. Each of us request Google (and same URLs) every 50 seconde, et Google doesn't block me.

BUT, if I add third device that do the same thing with my python script, Google show me the captcha on every device.
Could someone told me why Google don't show me the captcha when 2 devices with the same IP are doing same requests every 50 seconde, but when only 1 device is running with 1 request /40 seconde, it show me the captcha please ?

Google know how to recognize a device when we do python HTTP requests ?
Thank you very much
 
My guess is that when you run two devices (phone and computer) you are connected to mobile network through the phone and home IP through your home network? Would make most sense why third device triggers captcha since it would be 2nd on home network doing the same repetitive thing.

1 device for 1 request every 40 seconds trigger captcha is probably their limit for how long between each request at least. Sounds similar to Instagram botting issues where you needed to have not more than 2 accounts per IP and had to space actions out.

Are you sending out requests with a random number between 50 and 55?
 
Back
Top