Yeah, and I’d probably add a small cooldown for failed proxies too.
With random.choice() the same dead IP can get picked again almost immediately. Even a basic fail count and temporary skip would save a lot of pointless retries
Camoufox looks like the obvious one to try first
Just changing the user agent won’t do much on its own though. Proxy location, timezone, WebRTC and screen size should all match or the profile can still look weird
Using a full browser for every Google search sounds expensive for this.
I’d try a simple SERP parser first and only use Playwright when the normal request fails. Probably a lot cheaper than sending everything through Apify
If both tools stop after around the same number of searches, I’d check the proxies first.
Try a small run with a different list and watch what happens right before it stops. That should at least show whether it’s the proxies or ScrapeBox itself
For keyword scraping I'd still stick with datacenter unless Google is giving you trouble every single time.
Rotating helps, but only if the pool is actually large enough. Cycling through the same handful of IPs usually doesn't change much
This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.