Scraping..
- Thread starter innosoft
- Start date
Try using different proxies. I'm using 10 private ones from proxyblaze and my SB is going 24/7 with no hiccups. So far, no problems with them.
-I've tried squid proxies but they would die 3-4 days after and would need them replaced. Replacement would take 1-2 days.
-Tried from awesomenick but his support was a bit slow and they would die after about 15-20 days and would take 3-4 days to get them replaced.
In no way am I advertising for proxyblaze but just giving you my experience with different proxy providers. Hope you find something that works for you!
-I've tried squid proxies but they would die 3-4 days after and would need them replaced. Replacement would take 1-2 days.
-Tried from awesomenick but his support was a bit slow and they would die after about 15-20 days and would take 3-4 days to get them replaced.
In no way am I advertising for proxyblaze but just giving you my experience with different proxy providers. Hope you find something that works for you!
Tenshisendo
Registered Member
- Nov 20, 2010
- 64
- 24
You are running way to many threads I have 50 Private Proxies scraping with 20 threads for google only and am scraping roughly a million urls a day. I think.. I don't check everyday it usually runs for a week then I load up new footprints and keywords.
If for some reason you do block your ip in google it will come back in a day or so.
To stay under the radar I usually run 1/3 threads-proxy ratio.
- Nov 2, 2008
- 59,076
- 26,605