I have a feature request. Kindly add the Dupremove addon on scrapebox automator addon list. I know the automator has a similar function for harvester . But when the link list is huge, the harvester cannot load the lists. For me, I need to load a lists of 8GB and then split it.
the harvester handles 8Gb lists fine for me. Does it crash saying out of memory or ? The harvester grid in Scrapebox should support up to something like 134 million lines I think.
The latest version is bugged. When you scrape it will get a bunch of errors, skip to around 10% done, then crash.
It's 100% not a proxy problem, because when you revert to 2.0.0.73 it works fine.
Not really interested at this point going through support and giving detailed info, as there are obviously enough people in this thread having the same problem and doing that.
Either way you guys should take a look and fix it, since the harvester is pretty much the main point of Scrapebox
They are already working on it. Basically they rebuilt a lot of the thread code in .72 to make proxies work exponentially better and avoid bans better then ever.
Well a few users have an issue with it, so they built .74 to try and fix it, but the harvester is extremely complex. So sometimes when you change 1 thing it cascades into something else. So the .74 was an attempt to fix the issues that a few users were having with .72 and .73. It even says in the change log that it "might" be fixed. This is all standard, taking a few versions to sort out any bugs for major changes.
I tested the new version and it was fine for me and them and they released it and it works for like 99% of people.
Anyway, you can tell they are working on it because the download link for Scrapebox
http://www.scrapebox.com/payment-received
Will always have the latest version, but right now it still have .71 even though .74 is the latest update. That way if someone is having an issue they can go download .71.
Also in help >> restore previous version - you can roll back, which is sounds like what you did. So if .73 is working go with it, else download .71 at the above link and when .75 comes out, try that.
The improvements include the ability to change how often a proxy is rotated. So you can set it to rotate on every request or every 25th request or anything in between. This is huge, because as you use various engines or build in your own you can find the sweet spot and set it and it can dramatically affect ip blocks. AFter all avoiding those and still harvesting as fast as possible is the goal.
I've replied your pm..Kindly check. Thanks
thanks mate, pmed you back.