Permanently Closed Marketplace Sales Thread

[Sweetfunny]

With the actual stats I have no insight what is happening. There isnt even a log after stopping. And I dont know if google was requested for each keyword and keyword result page...

Maybe im a perfectionist and want to have all results that are available... but at the moment the multiharvester gives no insight what its doing. Thats not good for me...

Ok here's the thing right, the multi-threaded harvester was written totally from scratch it wasn't simply adding a bunch of thread to the old one it's a total ground up rebuild.

Here is Sciborg's screenshot scraping at 2,133 URL's per second when it was set to 100 connections, and since it's been upped to 500 connections and i've seen just over 4,000 URL's scraped per second (1/4 Million URL's in 60 seconds).

Granted i've only ever used a few scrapers, but i've never seen numbers like that before so ScrapeBox is probably one of the fastest scrapers that exist. Doing that in about 3 days, plus piling on granular stats when the kinks are still being ironed out wouldn't be wise there would of been complaints everywhere and problem tracing would be a nightmare.

As you seen there was a socket timeout issue, IPC getting swamped etc and this is why the old harvester is still available in tandem.

There's numerous problems to solve when reporting what 2k threads are doing, in different states across 4 engines.

<insert Chinese proverb about patience>

 

[danielgb123]

Is it weird that when I am harvesting I have over 50% of my links removed because the keywords are similar?
For instance I just had 91% removed which is around 6,000

 

[jibran_pcc]

Please change the havester all the links get delete for duplication and they are not duplicated:S. Its not working now...

 

[Sweetfunny]

Please change the havester all the links get delete for duplication and they are not duplicated:S. Its not working now...

"Options" then untick "Automatically Remove Duplicate Domains" and change it yourself.

 

[cdc515]

Suggestions:

1. in the harvestor can you add a progress bar i never know how much longer it is going to take to finish the scraping.

2. can you add in the harvestor a way that we can add a list of keywords and a list of footprints.

Love the new updates by the way.

 

[webmarketingmaster]

Is it weird that when I am harvesting I have over 50% of my links removed because the keywords are similar?
For instance I just had 91% removed which is around 6,000

I believe it is removing duplicate URL's automatically. Look at the changelog under help and you will see that this was added is this new update.

SB is suggesting that you not use so many similar keywords... and you wont get so many dulicate URL's. You dont need dulicate URL's so this shouldnt be a problem that it is removing them for you.

 

[softtouch2009]

SB removes automatically (when checked in the Options menu) duplicate domains, similar when you click the button "Remove Duplicates" and select "Remove Duplicate Domains".

 

[kendra]

Automatically removing duplicates is a good idea. I am sure there were many people posting away to the full list of scraped URLs without checking for duplication. The automatic duplication removal compensates for such peoples stupidity - meaning less idiotic spam on blogs, which is better for all of us.

 

[The Captain]

If your problems are when using the multithreaded harvester, it is also happening to me with my SB's.

Me too, but it only seems to be happening with google

 

[Sweetfunny]

Automatically removing duplicates is a good idea. I am sure there were many people posting away to the full list of scraped URLs without checking for duplication. The automatic duplication removal compensates for such peoples stupidity - meaning less idiotic spam on blogs, which is better for all of us.

Yes exactly, dealing with peoples "ScrapeBox doesn't work" support emails every day for 5 months now i have lost count the amount of times people were trying to comment to the same blog over 1,000 times in 2 minutes which helps nobody.

It gets paid proxy providers shut down due to complaints, causes blog owners to remove comment forms entirely, causes me a load of emails and generally makes things worse for everyone.

Unfortunately it had to be done, but of course the foam padding on the sharp edges can be removed by the knowledgeable with a few clicks.

 

[danielgb123]

I was wondering do a lot of errors come up due to the crappy free proxies?
While running the
linkdomain:yoururl.com

I continue to get error messages that close the program - I've sent the error support thing, so hopefully it's fixed.

 

[designsonline]

I had been using the scrape to harvest all the URLs from a specific website using site:xxx.com and then sorting the results by PR, but in the latest update it then strips all the results from the harvester and just gives you the root domain and the sub domains, this is no good, please can we have an option to use the harvester without it automatically stripping the results out from under a domain. This is one of the main things I use the program for, it is really useful for finding relevant high PR pages to link from.

Please add the option back in, or if it is already there please point out how I can find it.

Edit - There is already this option, I have found it under Options>untick automatically remove duplicate domains

(on a side note, are there any rough guides on how to use the free plugins?)

 

[jaybird]

Thanks for the updates Sweetfunny!

The MultiThreaded Harvester rocks! I had a crash or two, but it seems fine now as I just harvested 45,000 blogs in under 4 minutes!

Keep up the good work, but don't let the support suck the life out of ya!

This should make the "IM Product of the year" just for its scrape capabilities alone, much less the other features... I'm about to setup another dedicated machine ...

Jaybird :240:

 

[h4ckzor3]

Hello Sweetfunny,

Can you help me out with a question ?

I would like to use the target blog url in my comments but when i do %BLOGURL% it goes for full link including the article.
If i trim to root, i can`t make comments inside the needed article.
I believe that if i include the website url in my comments the post looks actually natural to the webmaster.


So, How can i do this type of comment ?

Thanks in advance !


PS: great tool people ! 57$ is a bargain.

 

[markiid]

The new multithreaded harvester is a beast. I gave it a small test on my vps and set it at 200 connections per search engine. This is the results I got.

Im gonna give it a go with 500 connections and see how it goes .

Great job Sweetfunny.

 

[proxygo]

same here since the update
im getting better test results on
tested proxies 6-800 per test.
excellent work sweet.

 

[jaybird]

Hello Sweetfunny,

Can you help me out with a question ?

I would like to use the target blog url in my comments but when i do %BLOGURL% it goes for full link including the article.
If i trim to root, i can`t make comments inside the needed article.
I believe that if i include the website url in my comments the post looks actually natural to the webmaster.


So, How can i do this type of comment ?

Thanks in advance !


PS: great tool people ! 57$ is a bargain.

The variable will not give you the root domain ... Sweetfunny would have to add another variable such as %BLOGROOTURL% or just %BLOGROOT% or even %!BLOGURL% with an exclamation

But I hope we get to keep the %BLOGURL% since it is great when used correctly, I would hate to lose that variable.

Jaybird

 

[Sweetfunny]

The new multithreaded harvester is a beast. I gave it a small test on my vps and set it at 200 connections per search engine. This is the results I got.

Im gonna give it a go with 500 connections and see how it goes .

Great job Sweetfunny.

Damn, that's roughly 300,000 URL's per minute... How much smoke is coming out of your network card?

Thanks for the feedback, it's great to see how well it's performing.

h4ckzor3 yes that will need another variable.

 

[Dark Knight]

oops After this update my sb doesn't scrape any url wtf

 

[mdzarate]

What do I need to do to use the new multi thread?