Question about Scrapebox Havester Limit.

[spectrejoe]

What's the limit on the harvester before scrapebox crashes on me when exporting and shit?

I'm currently running it for 7 hours and harvested 1 million URLS

 

[legoknekt]

It happens from time to time, but not so often, Have you tried to split the links in to smaller batches?

And check the harvester sessions folder btw, might be there.

 

[spectrejoe]

It happens from time to time, but not so often, Have you tried to split the links in to smaller batches?

And check the harvester sessions folder btw, might be there.

I can only split after exporting from the harvester not while im still harvesting.

I don't know at what number I should stop the harvester. It's only at 7% and already scraped 1 million urls

 

[The SEO]

When you try to get the unique URLs they will be just 50,000 to 100K out of 1000K.

Don't worry, all the scraping URLs also stored in a file in your ScrapeBox folder must be named as Harvesting Sessions, retrieve the list from there.

 

[spectrejoe]

When you try to get the unique URLs they will be just 50,000 to 100K out of 1000K.

Don't worry, all the scraping URLs also stored in a file in your ScrapeBox folder must be named as Harvesting Sessions, retrieve the list from there.

So I can harvest 10M and its ok?

 

[The SEO]

If they are web 2.0 sites then let him go beyond 10M until reached at least above 70%.

 

[spectrejoe]

If they are web 2.0 sites then let him go beyond 10M until reached at least above 70%.

blog comment links using GSA footprints

 

[loopline]

What's the limit on the harvester before scrapebox crashes on me when exporting and shit?

I'm currently running it for 7 hours and harvested 1 million URLS

depends on your memory of your machine, assuming your using 64bit. Then I would say maybe a billion or 2 billion, yes BILLION should be fine.

I definitely work with hundreds of millions at times.

Scrapebox V2 is 64bit so as long as your using the 64bit and your machine is 64bit then the limit is the amount of memory you have in your machine. But yes Scrapebox can handle 10 million no problem, unless your running really low memory then maybe not.

Plus even if you run out of memory, the results are saved in the harvester session folder in real time so you don't actually lose them.

 

[spectrejoe]

depends on your memory of your machine, assuming your using 64bit. Then I would say maybe a billion or 2 billion, yes BILLION should be fine.

I definitely work with hundreds of millions at times.

Scrapebox V2 is 64bit so as long as your using the 64bit and your machine is 64bit then the limit is the amount of memory you have in your machine. But yes Scrapebox can handle 10 million no problem, unless your running really low memory then maybe not.

Plus even if you run out of memory, the results are saved in the harvester session folder in real time so you don't actually lose them.

Jesus those are some scary numbers o.o

But thanks

 

[loopline]

Haha your welcome.

Go Big or Go Home.

 

[seedhano]

Scrapebox is powerful and the fact that they automatically store all your harvested urls on your computer does not cause you to lose anything in the event of a crash.