Storm Proxies and Rotating Proxies Not Working For Scrapebox?

[NikolaKica]

Guys,
Any idea about Storm Proxies and Rotating Proxies not working for Scrapebox?
I tried Storm Proxies but they didn't work. Then I got Rotating Proxies -
Couldn't harvest for a day - then I was able to harvest for 2 days - now I'm not able to harvest anymore.
Have no idea what's the problem.
Tried all sorts of solutions I could find on google/youtube nothing seems to work.
Is there a problem with those providers these days?
Any recommendations I can get so I can start scraping immediately? Without problem?
What are you using? I need proxies ASAP.

Thanks

 

[710fla]

Guys,
Any idea about Storm Proxies and Rotating Proxies not working for Scrapebox?
I tried Storm Proxies but they didn't work. Then I got Rotating Proxies -
Couldn't harvest for a day - then I was able to harvest for 2 days - now I'm not able to harvest anymore.
Have no idea what's the problem.
Tried all sorts of solutions I could find on google/youtube nothing seems to work.
Is there a problem with those providers these days?
Any recommendations I can get so I can start scraping immediately? Without problem?
What are you using? I need proxies ASAP.

Thanks

You scraping Google in Scrapebox?

 

[NikolaKica]

You scraping Google in Scrapebox?

I scrape bing yahoo too, but I need google also.
Bing and Yahoo work without proxies.
Yahoo works with proxies.
Google doesn't work.

 

[loopline]

Guys,
Any idea about Storm Proxies and Rotating Proxies not working for Scrapebox?
I tried Storm Proxies but they didn't work. Then I got Rotating Proxies -
Couldn't harvest for a day - then I was able to harvest for 2 days - now I'm not able to harvest anymore.
Have no idea what's the problem.
Tried all sorts of solutions I could find on google/youtube nothing seems to work.
Is there a problem with those providers these days?
Any recommendations I can get so I can start scraping immediately? Without problem?
What are you using? I need proxies ASAP.

Thanks

Did you try giving your machine a swift kick? Won't help but probably make you feel better. lol

Are you getting errors in the errors column?

The issue is possibly not cut and dry.

1 - go to settings >> harvester engine configureation >> import >> download default engines

try it, if it works, great.

2 - if it doesn't work then go to help >> show error log >> harvester - what are the errors?

503, 429 and 302 are all ip bans from google. With a huge pool of proxies, others can get the ips banned and google bans faster then ever these days.

3 - are you using advanced operators or queries? Try some basic queries as a test, like

car
cloud
green
etc...

do those work?

4 max out your proxy retries under settings >> connections timeouts and other settings >> more harvester options - proxy retries.

5 - wish this wasn't the case, but google has started the dreaded splash screens like bing and yahoo in some locations. Forcing you to accept tos service/privacy stuff before you proceed and scrapebox can not get around these. You will know this is the case if you follow the above and you get zero results, but there are no errors in teh errors log/errors column when harvesting.

You need ips from a country that doesn't have laws that require these types of splash screens. Think GDPR and similar, Ive noticed it for many countries in Europe and around and some countries in South America, but there could be others.

6 - If you made it thru 1-5 give your self a pat on the back for good effort and stick with it, you will get it.


Edit:

7 - something else you can try is go to settings >> harvester engine configuration.

the click google and at the bottom there is a test engine button and also an option to add in your proxy. So put in the rotating proxy gateway.

do the test, if it fails, save the raw html. Do that a half dozen times. Don't worry about if it works a couple times, we are looking for the failures here.

then open that raw html in a browser. What do you see?

Is it a lot of pages that have like a blank white square in the center of the page, possibly with a spinning circle still loading type thing? This is the splash page I was talking about

or is it a page that says there is bot traffic or similar?

or does it look normal? Because its possible your getting some ip from like malyasia or wherever (doesn't matter the country, just picked on at random), that is redirecting to the local google and that google has different html. Thats not typical for google, but it is possible. Worst case send these tests to support and they can tell you more if you can't sort it.

 

[NikolaKica]

Did you try giving your machine a swift kick? Won't help but probably make you feel better. lol

Are you getting errors in the errors column?

The issue is possibly not cut and dry.

1 - go to settings >> harvester engine configureation >> import >> download default engines

try it, if it works, great.

2 - if it doesn't work then go to help >> show error log >> harvester - what are the errors?

503, 429 and 302 are all ip bans from google. With a huge pool of proxies, others can get the ips banned and google bans faster then ever these days.

3 - are you using advanced operators or queries? Try some basic queries as a test, like

car
cloud
green
etc...

do those work?

4 max out your proxy retries under settings >> connections timeouts and other settings >> more harvester options - proxy retries.

5 - wish this wasn't the case, but google has started the dreaded splash screens like bing and yahoo in some locations. Forcing you to accept tos service/privacy stuff before you proceed and scrapebox can not get around these. You will know this is the case if you follow the above and you get zero results, but there are no errors in teh errors log/errors column when harvesting.

You need ips from a country that doesn't have laws that require these types of splash screens. Think GDPR and similar, Ive noticed it for many countries in Europe and around and some countries in South America, but there could be others.

6 - If you made it thru 1-5 give your self a pat on the back for good effort and stick with it, you will get it.


Edit:

7 - something else you can try is go to settings >> harvester engine configuration.

the click google and at the bottom there is a test engine button and also an option to add in your proxy. So put in the rotating proxy gateway.

do the test, if it fails, save the raw html. Do that a half dozen times. Don't worry about if it works a couple times, we are looking for the failures here.

then open that raw html in a browser. What do you see?

Is it a lot of pages that have like a blank white square in the center of the page, possibly with a spinning circle still loading type thing? This is the splash page I was talking about

or is it a page that says there is bot traffic or similar?

or does it look normal? Because its possible your getting some ip from like malyasia or wherever (doesn't matter the country, just picked on at random), that is redirecting to the local google and that google has different html. Thats not typical for google, but it is possible. Worst case send these tests to support and they can tell you more if you can't sort it.

I told my coworker to call the virtual machine host and ask him to kick the computer because we aren’t there. So yeah I understand the old russian method!
So, I’ve tried 1-4 and nothing works. You’re telling me I need to get proxies from a country that doesn’t have GDPR laws? What proxies do you guys use if I might ask? Or what proxie provider can I ask for those type of proxies? If you could send me a pm if that’s not a problem that would be awesome! Apart from that I see the 7th step as I’m writing the post. I’ll do that as soon as I’m done scraping emails tomorrow. I’m using a private list atm. No error logs in the harvester btw.
Will check the 7th option!
Thanks
Nikola

 

[turelink]

Those IPs should be blocked by Google!
Set time-out even rotating...for Each port, just adds limited proxies...

 

[NikolaKica]

Those IPs should be blocked by Google!
Set time-out even rotating...for Each port, just adds limited proxies...

Wish I knew what you said.

 

[NikolaKica]

Did you try giving your machine a swift kick? Won't help but probably make you feel better. lol

Are you getting errors in the errors column?

The issue is possibly not cut and dry.

1 - go to settings >> harvester engine configureation >> import >> download default engines

try it, if it works, great.

2 - if it doesn't work then go to help >> show error log >> harvester - what are the errors?

503, 429 and 302 are all ip bans from google. With a huge pool of proxies, others can get the ips banned and google bans faster then ever these days.

3 - are you using advanced operators or queries? Try some basic queries as a test, like

car
cloud
green
etc...

do those work?

4 max out your proxy retries under settings >> connections timeouts and other settings >> more harvester options - proxy retries.

5 - wish this wasn't the case, but google has started the dreaded splash screens like bing and yahoo in some locations. Forcing you to accept tos service/privacy stuff before you proceed and scrapebox can not get around these. You will know this is the case if you follow the above and you get zero results, but there are no errors in teh errors log/errors column when harvesting.

You need ips from a country that doesn't have laws that require these types of splash screens. Think GDPR and similar, Ive noticed it for many countries in Europe and around and some countries in South America, but there could be others.

6 - If you made it thru 1-5 give your self a pat on the back for good effort and stick with it, you will get it.


Edit:

7 - something else you can try is go to settings >> harvester engine configuration.

the click google and at the bottom there is a test engine button and also an option to add in your proxy. So put in the rotating proxy gateway.

do the test, if it fails, save the raw html. Do that a half dozen times. Don't worry about if it works a couple times, we are looking for the failures here.

then open that raw html in a browser. What do you see?

Is it a lot of pages that have like a blank white square in the center of the page, possibly with a spinning circle still loading type thing? This is the splash page I was talking about

or is it a page that says there is bot traffic or similar?

or does it look normal? Because its possible your getting some ip from like malyasia or wherever (doesn't matter the country, just picked on at random), that is redirecting to the local google and that google has different html. Thats not typical for google, but it is possible. Worst case send these tests to support and they can tell you more if you can't sort it.

Tried the 7th step - it says when I use the proxy:
No links could be retrieved!
403 HTTP/1.1 403 Forbidden


When I use the test without proxies and says it retrieved links:
When I open them in a browser - google chrome gets normal results: a search query
Internet explorer can't open them - it says trying to redirect.
But with Proxies I get the above error and can't even save the HTML code - it blocks it on the step earlier - doesn't even open that harvesting windows.
I assume it's the IP's. Any company suggestion? And what to ask them off? I need something most people don't have problem with. Backrotating proxies not working so far.

Thanks.

Will contact your support also.

Thanks!
Nikola!

 

[loopline]

Tried the 7th step - it says when I use the proxy:
No links could be retrieved!
403 HTTP/1.1 403 Forbidden

View attachment 155965
When I use the test without proxies and says it retrieved links:
When I open them in a browser - google chrome gets normal results: a search query
Internet explorer can't open them - it says trying to redirect.
But with Proxies I get the above error and can't even save the HTML code - it blocks it on the step earlier - doesn't even open that harvesting windows.
I assume it's the IP's. Any company suggestion? And what to ask them off? I need something most people don't have problem with. Backrotating proxies not working so far.

Thanks.

Will contact your support also.

Thanks!
Nikola!

Yes, if its a 403, 429 or 503 from google there is not really going to be any html to save.

That said, 403 is not an error I typically see from google, but rather proxies. Did you test the proxies, do they work? Its possible google is now returning 403 as well, they change things up all the time.

But your looking at blocked proxies. So if this is a rotating proxy service, you can try the detailed harvester, it is single threaded, but it has endless retries. So as a test its worth it to see if you can get results or not. Assuming your on windows.

else then you can try a different provider, but with advanced footprints, finding a rotating provider that works, might be near impossible.

 

[710fla]

Yes, if its a 403, 429 or 503 from google there is not really going to be any html to save.

That said, 403 is not an error I typically see from google, but rather proxies. Did you test the proxies, do they work? Its possible google is now returning 403 as well, they change things up all the time.

But your looking at blocked proxies. So if this is a rotating proxy service, you can try the detailed harvester, it is single threaded, but it has endless retries. So as a test its worth it to see if you can get results or not. Assuming your on windows.

else then you can try a different provider, but with advanced footprints, finding a rotating provider that works, might be near impossible.

Was using rotating proxies.com after seeing you recommend them but they no longer work to scrape Google. Any other suggestions?

 

[loopline]

Was using rotating proxies.com after seeing you recommend them but they no longer work to scrape Google. Any other suggestions?

No not at the moment. Its a tough game. Ive seen a lot of people metioning scraper api and then a lot of people complaining they don't work with google either, so...

1 problem is any service I recommend gets flooded and then starts to not work as well, lol. Storm proxies held out for like 2 or 3 years of me recomending them, but then google changed and storm proxies did not.

For a few years I quit recomending rotating providers because every time I would find one that worked, I would tell people, and then it would quit working and I couldn't use it any more either. that worked for a while and by not telling people I could keep using a service, but... I feel bad not telling people. So these days I don't hold anything back.

What I can also say is if you can get your footprints to work with bing, they are much more forgiving then google is.

Otherwise its a bit of a chase to find the latest and greatest.

 

[NikolaKica]

No not at the moment. Its a tough game. Ive seen a lot of people metioning scraper api and then a lot of people complaining they don't work with google either, so...

1 problem is any service I recommend gets flooded and then starts to not work as well, lol. Storm proxies held out for like 2 or 3 years of me recomending them, but then google changed and storm proxies did not.

For a few years I quit recomending rotating providers because every time I would find one that worked, I would tell people, and then it would quit working and I couldn't use it any more either. that worked for a while and by not telling people I could keep using a service, but... I feel bad not telling people. So these days I don't hold anything back.

What I can also say is if you can get your footprints to work with bing, they are much more forgiving then google is.

Otherwise its a bit of a chase to find the latest and greatest.

ugh. will find one and be quiet about them

 

[fravel]

I deactivated my subscription after 3 months of use. Not reliable for scrapping Google and the proxy is blacklisted on some websites.

Despite having taken all precautions, renewed the gateways. I even provided the support with screen prints of my settings.

Support couldn't do anything even if they were reactive they stop answering me when I sent them my error logs (a lot of 429 Too many requests errors).

 

[NikolaKica]

I deactivated my subscription after 3 months of use. Not reliable for scrapping Google and the proxy is blacklisted on some websites.

Despite having taken all precautions, renewed the gateways. I even provided the support with screen prints of my settings.

Support couldn't do anything even if they were reactive they stop answering me when I sent them my error logs (a lot of 429 Too many requests errors).

Thanks

 

[ristoriel]

I don't have a first hand experience with Scrapebox so I'm sure how elaborate "proxies not working for scrapebox" is. Perhaps 4G mobile proxies would be better than datacenter? If it is a problem on your proxy provider's end and you're looking for a new provider, try checking the BHW proxies marketplace.

 

[loutiop]

Maybe you should try other proxy providers in the marketplace

 

[dagar]

I scrape bing yahoo too, but I need google also.
Bing and Yahoo work without proxies.
Yahoo works with proxies.
Google doesn't work.

No not at the moment. Its a tough game. Ive seen a lot of people metioning scraper api and then a lot of people complaining they don't work with google either, so...

1 problem is any service I recommend gets flooded and then starts to not work as well, lol. Storm proxies held out for like 2 or 3 years of me recomending them, but then google changed and storm proxies did not.

For a few years I quit recomending rotating providers because every time I would find one that worked, I would tell people, and then it would quit working and I couldn't use it any more either. that worked for a while and by not telling people I could keep using a service, but... I feel bad not telling people. So these days I don't hold anything back.

What I can also say is if you can get your footprints to work with bing, they are much more forgiving then google is.

Otherwise its a bit of a chase to find the latest and greatest.

see i harvested many url with my proxy

 

[proxygo]

posting scraping result of 1 proxy is hardly any result at all

 

[NikolaKica]

posting scraping result of 1 proxy is hardly any result at all

Maybe he uses rotating proxies and that's only the servers proxy/or how you'd say that.