- Aug 25, 2015
- 5,523
- 2,859
I emailed stormproxies with the issue and they replaced my gateway. Did they do the same?
Scrapebox url scrap error 429
- Thread starter xfreedom
- Start date
I emailed stormproxies with the issue and they replaced my gateway. Did they do the same?
Paco Adorno
Newbie
- Jan 13, 2020
- 7
- 2
Thanks again! Yes, it's definitely worst when I use advanced operators. If I don't use advanced operators l get around 50 url/s but once I add them it goes down to around 10. I'll keep trying. Hopefully it's something they can address. Maybe I'll just have to upgrade the threads package. Even their highest tier is cheaper than most services. Thanks for the help.
Paco Adorno
Newbie
- Jan 13, 2020
- 7
- 2
They did not. Did it do the trick? They told me it could be something about my software's signature. Like Google could detect scrapebox' signature even with proxies. I googled that to see if it's a thing but didn't find anything. But please let me know if replacing the gateway worked. Thanks!
- Aug 25, 2015
- 5,523
- 2,859
Still lots of errors. They'll probably get it fixed soon if lots of people are having the same problem.
Still able to scrape but lots of errors, more than before barely got any errors
Paco Adorno
Newbie
- Jan 13, 2020
- 7
- 2
Same here. Lots of errors and a lot slower. Significantly slower when using advanced operators. Thanks for the reply. If I notice any changes, I'll post in the thread.
- Jan 25, 2009
- 6,378
- 3,774
I had 1 person today say that storm proxies refreshed their ip pool and they were now working. Ive no idea it storm proxies did something for just that one user or on the whole.
Its not a thing. Scrapebox uses raw sockets and threads and scrapebox does not execute javascript. If scrapebox executed javascript then that gives google power to see things, but by scrapebox not executing javascript google can not tell if its scrapebox or firefox or chrome etc.. as scrapebox uses those user agents and looks like a browser.
Paco Adorno
Newbie
- Jan 13, 2020
- 7
- 2
No improvement on my end. The last response I got from Storm Proxies was "this is because the change of last update in google that they have made more restriction in scraping." This response does not give me hope that they are doing anything about it. It seems like this is the new normal for them. Are other proxy services experiencing the same thing? Maybe it's time to switch? I'm going to ask them if I should expect this same performance going forward or if I should expect it to improve and I'll share that response.
If anyone can share their experience with other proxy providers I will appreciate it. Are they getting hit in the same way as storm proxies? Are they business as usual? Maybe they experienced an impact but not as bad as Storm Proxies? Any recommendations? I appreciate the help!
Thanks!
If anyone can share their experience with other proxy providers I will appreciate it. Are they getting hit in the same way as storm proxies? Are they business as usual? Maybe they experienced an impact but not as bad as Storm Proxies? Any recommendations? I appreciate the help!
Thanks!
- Jan 25, 2009
- 6,378
- 3,774
I don't know of any other great providers at the moment, but its too early to tell. I do not know if it will be the new norm for storm proxies, I hope not.
Im on the lookout for other providers, but nothing yet. If you find one, let us all know.
Paco Adorno
Newbie
- Jan 13, 2020
- 7
- 2
Thanks so much for sharing that! It's good to know that it's too early to tell. Hopefully things improve. Today I noticed the URL/Sec went up from 2 to 15 (using 1 advanced operator). I can live with 15. Let's see if this stays consistent. I'll probably do a few trials. I'll report back if I do. Thanks!
- Jan 25, 2009
- 6,378
- 3,774
Well thats definitely good. Worst case, and I hate to say it, if a bunch of people exit them and cancel their service over this, and they have the same amount of proxies, then that will inherantly make them work better for the customers who stay. But hopefully they can get more proxies.
the only real solutions are more proxies or less customers using them. Yes please report back, Id love to know how you get on.
I have reported my result using stormproxies on the other post:
https://www.blackhatworld.com/seo/scrapebox-issue.1190901/
From the result I think the problem is on the proxies. But, about 1/10 proxies from stormproxies still works.
So I switch to luminati.io and they give me 5 dollars to test their service. I choose to test their data center ips. The ip type is “shared (pay per usage)”, and its responding ip pool size is 20000. The result is worse than stormproxies.
But I noticed a weird thing. I use the luminati proxy as system proxy, visit google by browser and put in the url used by scrapebox. I found it works fine. I remember I did the same thing using stormproxies and the result is same. But I can’t figure out why.
Hope the report helps.
https://www.blackhatworld.com/seo/scrapebox-issue.1190901/
From the result I think the problem is on the proxies. But, about 1/10 proxies from stormproxies still works.
So I switch to luminati.io and they give me 5 dollars to test their service. I choose to test their data center ips. The ip type is “shared (pay per usage)”, and its responding ip pool size is 20000. The result is worse than stormproxies.
But I noticed a weird thing. I use the luminati proxy as system proxy, visit google by browser and put in the url used by scrapebox. I found it works fine. I remember I did the same thing using stormproxies and the result is same. But I can’t figure out why.
Hope the report helps.
- Jan 25, 2009
- 6,378
- 3,774
Javascript.
Google uses a mass of javascript and with that they can see around proxies a lot of times and they have the control. When you turn off javascript, like scrapebox does, google loses the control and then they ban ips faster.
- Jan 25, 2009
- 6,378
- 3,774
If its a back connect/rotating/reverse proxy provider, then its supposed to show that. So just don't test them, just use them.
Also a video for that type of proxy
stan-paris
Newbie
- Oct 12, 2020
- 1
- 0
Hello @loopline ,
First: thank you for your great tool and your perfect videos! Long time scrapebox user. Today the harvester does not give me any more results for Google.
I :
- reinstalled VPN &
- Scrapebox
I
- changed the IP of my VPN
- bought new proxies
- also took Storm's proxies.
I've also :
- Updated to the latest Harvester Engine Configuration
- Reduced the number of Harvester connections then increased the timeout.
Result:
>> 10/12/2020 6:31:14 AM: HTTP: 429 HTTP / 1.1 429 Too Many Requests, URL: https://www.google.fr/search?complete=0&hl=fr&q=ub&num=100&start=0&filter=0&pws = 0 Proxy: 45 ........
I do not know what to do. A test for me to understand where the problem comes from?
thank you in advance
First: thank you for your great tool and your perfect videos! Long time scrapebox user. Today the harvester does not give me any more results for Google.
I :
- reinstalled VPN &
- Scrapebox
I
- changed the IP of my VPN
- bought new proxies
- also took Storm's proxies.
I've also :
- Updated to the latest Harvester Engine Configuration
- Reduced the number of Harvester connections then increased the timeout.
Result:
>> 10/12/2020 6:31:14 AM: HTTP: 429 HTTP / 1.1 429 Too Many Requests, URL: https://www.google.fr/search?complete=0&hl=fr&q=ub&num=100&start=0&filter=0&pws = 0 Proxy: 45 ........
I do not know what to do. A test for me to understand where the problem comes from?
thank you in advance
- Jan 25, 2009
- 6,378
- 3,774
>> 10/12/2020 6:31:14 AM: HTTP: 429 HTTP / 1.1 429 Too Many Requests, URL: https://www.google.fr/search?complete=0&hl=fr&q=ub&num=100&start=0&filter=0&pws = 0 Proxy: 45 ........
means the ip(s) used were blocked. 429 is a block code from google.
So it sounds like maybe thats your issue. Also make sure your google engines are updated, settings >> harvester engine configuration >> import >> download default engines
But the pages were still loading before the engine update, so your 429 is a blocked ip.
jrengjreng
Registered Member
- Feb 28, 2018
- 72
- 14
i also unable to scrape from google..i use resident IP..yahoo in the other hand scraping as usual
- Sep 10, 2010
- 11,806
- 26,658
Are you doing site: queries? How many searches do you do per proxy? Asking this because you will even get recaptcha from browser if you do too many of those "dorks".
- Jan 25, 2009
- 6,378
- 3,774
Update your engines file and failing that its likely your proxies.
settings >> harvester engine configuration >> import >> download default engines.
Astraport
Regular Member
- Jul 9, 2016
- 292
- 42
Amazing. I use Scrapebox solely to check Google indexing of my urls. Everything works fine on my computer, but I transferred the license to the server and everything changed.
The first time the indexing check was successful, but then only this error: HTTP: 429 HTTP/1.1 429 Too Many Requests
I changed three proxy providers who had been doing this job just fine weeks earlier. I changed user agents, urls, timeouts. I tried everything and only Scrapebox remains, which for some reason does not want to work on the server.
The first time the indexing check was successful, but then only this error: HTTP: 429 HTTP/1.1 429 Too Many Requests
I changed three proxy providers who had been doing this job just fine weeks earlier. I changed user agents, urls, timeouts. I tried everything and only Scrapebox remains, which for some reason does not want to work on the server.