I understand what you are saying about using smaller scrapes - whilst practical it is not a viable solution for me.
For example with gscraper ( which is not my favorite scraper) I load a million keywords, and around 100 footprints and set to auto load proxies every x min. and i set to create a new save file every 1 million lines.
I use GSA Platform identifier and set it to monitor the Gscraper folder for new files.
This way i can scrape links and identify it and save it in a folder for GSA SER to use -- set and forget for 2 -3 months non stop.
With Scrapebox ( which is my favorite scraper ) it is a time waste, as I need to first scrape and wait for it to finish before i can do anything with the results. This might be practical on a small scale, say 1 keyword and 1 footprint , but not practical if you scale it up.
Everything else in Scrapebox runs great, and it is my go to tool, i only wish that in future you can consider an alternative saving method ( similar to gscraper)
I get what your saying, and Im helping you out here and bear in mind Im telling you how I successfully run my system knowing full well you will take it and sell a competing product with the info. None the less I believe in helping people more then I believe in making money. So here is what I do, you can use it or leave it. I don't get to decide on whether or not they add so Im just sharing my 2 cents.
If you take your 1 million keywords and break it down into say 1000 keywords or your choice number, that gives you 1000 files of 1000 keywords each (or keyword/footprint combos etc.. you get the point).
Stick those in a file, they will be numbered keywords1 keywords2, keywords3 etc... or whatever.
You create an automator job file that first calls a external script, which copies a random file from your folder of keywords, overwrites it to keywords1 and then deletes that file. So say it grabs keywords1, then it would copy the contents of keywords1 and save it in keywords-fixed.txt and then delete keywords1.
You run your harvest in the automator on a fixed file, like "keywords-fixed.txt" and then it can export to whatever folder you want and add a time stamp. Then you have platform identifier monitoring that folder for new files.
Then the automator loops and this time the external script picks up keywords2.txt and copies it to keywords-fixed.txt and then keeps going. In this way the only step you have had to add is adding an automated script and you get the full benefit of of the same feature your after, but with more advantages then if they do it your way.
Because then your working with small files, which is more effecient. You still get regular output for PI to monitor as well. It can still run for months. At any point you can dump in more keyword files, so you dont' have to wait until the 1 big one finishes, which makes it seemless to run all year long.
You can have the automator load proxies on each loop and/or load proxies ever X often as well.
Depending on your needs you could then do other things. I myself am up for the award of the worlds top 100 slowest programmers and I can build this script in python 10 mins. Or you could use batch files. Someone fiverr would probably make it for you for $5.
I won't go into the rest of my "system" but working in small batches has multiple other advantages and even if Scrapebox had the method you are proposing, I would still use the above method.
My 2 cents.
Hi I cannot move my scrapebox to the new server for days - already sent support email but no response. Can you hit me up on skype: erichon15
They don't use skype, you can try scrapeboxhelp {at) gmail [dot) com
