Use the multi-threaded harvester, and the Harvester Keyword Statistics dialog to remove the completed keywords, so you don't use them again in a harvesting run.
- Scrapebox > Settings > Use multi-threaded harvester
- After the scraping starts to slow down (I'm using the latest version v1.15.42 and it does not hang), then select the "Stop harvesting" button.
- The Harvester Keyword Statistics dialog appears:
Select : Export keywords > Export all not completed keywords to Keyword list.
- Load fresh proxies.
- Start harvesting again!
Thanks, so this is how people harvest 300 000+ auto-approve URLs?
It doesn't hang, but at some point I get no more results, because all the public proxies have been banned from Google.
Let's say if I use a software like Proxy Goblin, then I should stop the harvester each day, replace the burnt proxies with new, fresh proxies, and load the harvester again but only searching for unused keywords?
I think a very useful feature in Scrapebox would be if I could modify the proxy list on the fly, while the harvester is running.
There is already a Proxies folder inside the Scrapebox folder, with a proxies.txt file in it. So when I modify that proxies.txt file, Scrapebox should monitor that file and notice that I change the proxies even if the harvester is still running. Then Scrapebox should use that new proxy list, without me stopping the harvester.
In fact this feature would completely automate the harvesting process, since a public proxy finder like Proxy Goblin can save new proxies to a txt file automatically, on schedule.
So I could just load a huge list of keywords into Scrapebox, start Proxy Goblin, and start the Scrapebox harvesting. What would happen after that is Proxy Goblin would keep updating the proxy list on auto pilot, and the Scrapebox harvester could run for days, even for weeks without baby sitting!
Why Scrapebox doesn't have this feature?