Recommended Google Proxy/Connection settings for Scrapebox

Who can give a recommended Google Proxy/connection settings for Scrapebox?

I am scraping with US and Canada proxies with operator like -
cafe "city" inurl:contact.
I had my previous proxies all banned, by Google and Bing/Yahoo are giving so little returns (I used the operators on their search engines directly, where Google is giving 6,000 results, Bing and Yahoo are giving 30)
I really need working recommendations to work with Google.
@loopline , @arc323 , or others, any suggestions?
Thanks

Pam Danny said:

Who can give a recommended Google Proxy/connection settings for Scrapebox?

I am scraping with US and Canada proxies with operator like -
cafe "city" inurl:contact.
I had my previous proxies all banned, by Google and Bing/Yahoo are giving so little returns (I used the operators on their search engines directly, where Google is giving 6,000 results, Bing and Yahoo are giving 30)
I really need working recommendations to work with Google.
@loopline , @arc323 , or others, any suggestions?
Thanks

Click to expand...

Hi, there is no ideal proxy/connection setting but it is about kind of proxy you using. You need a large pool of ips

What kind of proxies you used?

Jimoz said:

Hi, there is no ideal proxy/connection setting but it is about kind of proxy you using. You need a large pool of ips

What kind of proxies you used?

Click to expand...

I am using 90 private proxies from Viper Proxies

Wanna know it too

Pam Danny said:

I am using 90 private proxies from Viper Proxies

Click to expand...

Private proxies doesnt work for scraping google anymore. I use mobile rotating proxies and works fine for google. Other proxies just dont work like before. Google block them or throws recaptcha.

Jimoz said:

Private proxies doesnt work for scraping google anymore. I use mobile rotating proxies and works fine for google. Other proxies just dont work like before.

Click to expand...

what proxy provider you use?

Jimoz said:

Private proxies doesnt work for scraping google anymore. I use mobile rotating proxies and works fine for google. Other proxies just dont work like before.

Click to expand...

Nice. Can you share your settings for mobile rotating proxies? I'll try to get them.
How many?
Connections?
Timeout?
Delay?

Flymo said:

what proxy provider you use?

Click to expand...

You can use any mobile rotating proxy provider here at marketplace. do let them know what you plan to do and they will help you

Pam Danny said:

Nice. Can you share your settings for mobile rotating proxies? I'll try to get them.
How many?
Connections?
Timeout?
Delay?

Click to expand...

Just ask for Mutli threading Mobile proxy. I use 40-50 Connections and all other defaults, It scrapes close to million url in an hour. I use a server with high speed internet to run the scrapebox. you could get lower results depending upon your internet speed.

Pam Danny said:

Who can give a recommended Google Proxy/connection settings for Scrapebox?

I am scraping with US and Canada proxies with operator like -
cafe "city" inurl:contact.
I had my previous proxies all banned, by Google and Bing/Yahoo are giving so little returns (I used the operators on their search engines directly, where Google is giving 6,000 results, Bing and Yahoo are giving 30)
I really need working recommendations to work with Google.
@loopline , @arc323 , or others, any suggestions?
Thanks

Click to expand...

You can either find a big pool of proxies, many if not most of which will be blocked for your query, as it uses advanced operators (Because everyone else wants to use pools of ips to scrape google too). So make sure you max out your proxy retries, I have a video on how to utilize scrapebox with ideal settings for this type of proxies. Video below




Or you can try to just use private or shared private proxies with a big delay. I have a video on that as well, below again, but this approach is about using a delay to slow it down enough you don't get blocked. Its going to take a LARGE delay for 2021 and 2022, not a ratio like in the video. So you want to use detailed harvester and just try a delay, but it depends on how many proxies you have. Start with at least a delay of 300 and you can go higher if you need. Then if all goes well start dialing back the delay until you get some ips blocked. Then you know your sweet spot of how fast you can go, until google changes the rules again, which they will. If you get ips blocked, just let them rest with no use for 48 hours on google and they should get unblocked.

The more proxies you have, the lower the delay. Just don't get any ideas that you can use 10 proxies with a 10 second delay, its not going to happen. You might need more like 30 proxies with a 300 delay, I don't know, it depends on many factors.






Each method has their advantages, the pool gives you more ips, but if most of them are blocked is that really an advantage? And you only get so many retries so you may need to rerun the failed over and over. Might take longer in the end then using private proxies, might not.

With private proxies you have the advantage of figuring out your sweet spot and then just letting it run and you should get all the results in 1 or worst 2 runs, so less hassle, but may take longer. How much time do you have? Which is right for you? It depends, you will have to do a little trial and error to find the way that works best for you.

Why are you people still using software from '99 where you need proxies to scrape google?

https://ohyicong.medium.com/how-to-bypass-recaptcha-with-python-1d77a87a00d7 there's a github link in there, follow it. It will make your day great again.