Scrapebox Question - Pre-Caution to Avoid Ban

[SupremeBoner]

I've got 10 dedicated private proxies and am trying to scrape a vast amount of urls from google.

Inputing about 200-500 keywords per scrape, which after merging footprints ends up being about 5-10k total keywords. Im using multiple operaters per query (site:, intitle:, -2001 -2002, -2003, -2004) and also having 1000 results displayed per kw.

Haven't run SB yet because I am trying to figure out if I will need more proxies for this or if I can get away with it with 10, provided I set the right settings.

Currently I have RND Delay Range at 30s min-40s max, timeout of 75seconds, and running single thread harvester.

Will this be sufficient to aboid ban or will I need another 10 proxies? thanks

 

[loopline]

It depends. The more often a proxy is banned, the more easily it gets banned. The more often it gets banned the longer a ban lasts for. So your delay is "probably" ok so long as you go to settings and uncheck the use multi threaded harvester and use custom harvester. That way you are using the single threaded harvester (the only harvester that works with the delay).

 

[BondaruBond]

I want to buy SB. Where can I find a promo price?
Thk

 

[vortez]

Search for scrapebox discount on google

 

[StormClaw]

I want to buy SB. Where can I find a promo price?
Thk

kingwarrior.co

This guy owns loads of discounts for all IM tools and you can find the Best deal for SB there..

 

[SupremeBoner]

It depends. The more often a proxy is banned, the more easily it gets banned. The more often it gets banned the longer a ban lasts for. So your delay is "probably" ok so long as you go to settings and uncheck the use multi threaded harvester and use custom harvester. That way you are using the single threaded harvester (the only harvester that works with the delay).

Okay thanks. Is this the kind of thing you'd really recommend 20 proxies for though? Ill be running it pretty much continuously for a month, probably 5k keywords per harvest just to break things up in case my computer BSOD's or something. I imagine the url output collectively will be in the millions.

I ask about requiring 20 also because I'm assuming if majority of the proxies get banned mid-harvest you're kind of screwed.

Thanks

Edit: After about 50 pages 4 proxies were already 302 banned

 

[DemmyCool]

Cheapest Scrapebox Here: http://kingwarrior.co/scrapebox-discount only $52 bucks. He also gives some bonuses, don't forget to request them.
I have personally buy from this guy Scrapebox and CaptchaSniper so i know he's a good one fellow. Regards.

 

[loopline]

Okay thanks. Is this the kind of thing you'd really recommend 20 proxies for though? Ill be running it pretty much continuously for a month, probably 5k keywords per harvest just to break things up in case my computer BSOD's or something. I imagine the url output collectively will be in the millions.

I ask about requiring 20 also because I'm assuming if majority of the proxies get banned mid-harvest you're kind of screwed.

Thanks

Edit: After about 50 pages 4 proxies were already 302 banned

20 isn't enough without a delay. So its not about number of proxies, its about speed. If you want to go faster, you need more. 20 is fine, just add a 3-5 second delay.

What version of SB are you using?

 

[SupremeBoner]

20 isn't enough without a delay. So its not about number of proxies, its about speed. If you want to go faster, you need more. 20 is fine, just add a 3-5 second delay.

What version of SB are you using?

I've got 20 proxies from squid now. When you say 3-5 second delay does that mean an RND Delay range of minimum 3 seconds, maximum 5 seconds? Currently I have it said to minimum 35 seconds maximum 55 seconds. I've selected RND for delay, using the single thread harvester (aka custom and multi-thread harvester unchecked), and 75 second timeout.

I'm running version 1.16.4

 

[cottonwolf]

just go to scrapebox.com/BHW . Don't know if you can get it cheaper elsewhere

 

[laur.laurix]

With this delay u will get ur proxies banned after few thousand urls scraped. If u want to scrape millions of urls i recommend 2 connections with minimum 35 sec delay. This way u can scrape 3 days without stopping.

I've got 20 proxies from squid now. When you say 3-5 second delay does that mean an RND Delay range of minimum 3 seconds, maximum 5 seconds? Currently I have it said to minimum 35 seconds maximum 55 seconds. I've selected RND for delay, using the single thread harvester (aka custom and multi-thread harvester unchecked), and 75 second timeout.

I'm running version 1.16.4

 

[loopline]

I've got 20 proxies from squid now. When you say 3-5 second delay does that mean an RND Delay range of minimum 3 seconds, maximum 5 seconds? Currently I have it said to minimum 35 seconds maximum 55 seconds. I've selected RND for delay, using the single thread harvester (aka custom and multi-thread harvester unchecked), and 75 second timeout.

I'm running version 1.16.4

Yeah, if your using 1 connection, and a 3-5 second delay you should be able to harvest indefinitely. But it all depends on your queries, how many times you have got the proxies blocked and how many times the people before you got them blocked, their geo local and all sorts of stuff. So you will have to test and see.

 

[SupremeBoner]

Yeah, if your using 1 connection, and a 3-5 second delay you should be able to harvest indefinitely. But it all depends on your queries, how many times you have got the proxies blocked and how many times the people before you got them blocked, their geo local and all sorts of stuff. So you will have to test and see.

Hey Loop, thanks for all your help so far, although heres what happened:

I put in about 3k keywords (including footprints). Ran single threaded harvester on google canada to display 1,000 results. Set the RNS way up to min 55secs - max 60secs.

Harvester stopped itself at only 10,000 urls scraped after about 80 minutes.

I really don't get what I'm doing wrong. 20 brand new private squid proxies with a massive delay and yet this is the second time it stopped after only an hour.

 

[ronin28]

bump for you..(and me...) and solutions on this issue ?

Hey Loop, thanks for all your help so far, although heres what happened:

I put in about 3k keywords (including footprints). Ran single threaded harvester on google canada to display 1,000 results. Set the RNS way up to min 55secs - max 60secs.

Harvester stopped itself at only 10,000 urls scraped after about 80 minutes.

I really don't get what I'm doing wrong. 20 brand new private squid proxies with a massive delay and yet this is the second time it stopped after only an hour.