Permanently Closed Marketplace Sales Thread
- Thread starter Sweetfunny
- Start date
- Status
Mr.Boss
Power Member
- Sep 5, 2012
- 529
- 170
I am trying to run Scrapbox Competition Finder with more than 1 thread and I have more than enough private proxies but no matter what number of connections I set it always runs in 1 thread. Can you please let me know how to increase the the number of threads.
- May 10, 2012
- 9,541
- 7,560
I am sure that this request has already been mentioned here, but could we get a progress bar?
I mean, when I am removing all the URLs from a file with 2.5 million URLs that are contained in the file with 5 million URLs, then yeh, it's going to take 48 hours plus, but I would really love to know at least the approximate ETA or at the very least the current progress. Thanks
I mean, when I am removing all the URLs from a file with 2.5 million URLs that are contained in the file with 5 million URLs, then yeh, it's going to take 48 hours plus, but I would really love to know at least the approximate ETA or at the very least the current progress. Thanks
- Jan 25, 2009
- 6,378
- 3,774
I am trying to run Scrapbox Competition Finder with more than 1 thread and I have more than enough private proxies but no matter what number of connections I set it always runs in 1 thread. Can you please let me know how to increase the the number of threads.
When you set a delay it will override connections and set it to 1. Point being that if you need a delay then you shouldn't be using more then 1 connection because you can simply lower the connections and then effectively accomplish the same thing. So just lower your connections and remove the delay. This is pretty much true of anything in scrapebox.
I am sure that this request has already been mentioned here, but could we get a progress bar?
I mean, when I am removing all the URLs from a file with 2.5 million URLs that are contained in the file with 5 million URLs, then yeh, it's going to take 48 hours plus, but I would really love to know at least the approximate ETA or at the very least the current progress. Thanks
Does it really take you that long? I can do it exponentially quicker.
For example if you split that 2.5 million file into 5 files of 500K each and import and compare them it will go pretty fast and then you will be done with the lot in less then an hour probably. You could even build an automator job for this.
But there is a scale, and when you start reachigng a threshold with comparing files, its like a bell curve, all is well to a point and then as file size increases some time increases dramatically. Its the nature of it. My numbers may not be exact, but you get the point. There is also a further gain in the fact that on the 2nd 500K file your going to be comparing against like 4.7 million urls instead of 5 million etc..
Great post you made on expired domains yesterday too, should have thanked you there, but was in a hurry.
- May 10, 2012
- 9,541
- 7,560
Well it's been more like 56 now.
And thank you, glad you like it! Well I did some comparing of, give or take, 100K domains against 4 million file and can't say that it was super fast. Took an hour or two, but I suppose that the other bigger master file is the bottleneck here. I fully understand what you are saying and I have a similar experience, but it never really did go that fast on my end. Thank you for the explanation
- Jan 25, 2009
- 6,378
- 3,774
Well it's been more like 56 now.
Well I did some comparing of, give or take, 100K domains against 4 million file and can't say that it was super fast. Took an hour or two, but I suppose that the other bigger master file is the bottleneck here. I fully understand what you are saying and I have a similar experience, but it never really did go that fast on my end. Thank you for the explanation
Wow thats intense. It could also be the diversity. You might have a really diverse list vs the lists I was comparing with were a lot the same so the deduplication went much faster.
One other thing you can try which might help is if you are comparing on a domain level and not on the specific url, on the list that is the "old" list that your comparing against - you could remove duplicate domains (if you haven't already) and then trim them to root. That way you are ditching 50% or more of the total characters that have to be dealt with, which generally helps, but may or may not do that great depending on your list.
Its good to know though, in truth I don't "regularly/constantly" compare lists that are that big, I try to avoid it when possible as it does take a while regardless.
Im sitting here trying to think of a better way to do it and I have 1 idea, but mostly drawing a blank. Ill let you know if I come up with something.
- May 10, 2012
- 9,541
- 7,560
Well I had to take the notebook today for 2 hour drive and filtering still wasn't finished so I somehow managed to make the battery last long enough to reach my destination. So yeh, it's still filtering.
Well actually, I am sure that at least half of the new list is contained within the old one I am comparing against. Of course, all the domains were deduped and trimmed to root. Not sure if I can go any faster at this point. I really just wanted some kind of a progress bar so I can estimate when it will finish give or take. I remember importing 3 million domains to availability checker tool within SB and import alone took 72 hours.
Let me know, sure, anything will help.
- Jan 25, 2009
- 6,378
- 3,774
Well I don't work for scrapebox but Ill pass it on to them, and they read the thread anyway.Well I had to take the notebook today for 2 hour drive and filtering still wasn't finished so I somehow managed to make the battery last long enough to reach my destination. So yeh, it's still filtering.
Well actually, I am sure that at least half of the new list is contained within the old one I am comparing against. Of course, all the domains were deduped and trimmed to root. Not sure if I can go any faster at this point. I really just wanted some kind of a progress bar so I can estimate when it will finish give or take. I remember importing 3 million domains to availability checker tool within SB and import alone took 72 hours.
Let me know, sure, anything will help.
I have one more idea though, have you tried:
http://pc-fitness.tripod.com/images/hammer computer.jpg
- May 10, 2012
- 9,541
- 7,560
softtouch2009
Senior Member
- Dec 2, 2009
- 1,064
- 257
Well I had to take the notebook today for 2 hour drive and filtering still wasn't finished so I somehow managed to make the battery last long enough to reach my destination. So yeh, it's still filtering.
Well actually, I am sure that at least half of the new list is contained within the old one I am comparing against. Of course, all the domains were deduped and trimmed to root. Not sure if I can go any faster at this point. I really just wanted some kind of a progress bar so I can estimate when it will finish give or take. I remember importing 3 million domains to availability checker tool within SB and import alone took 72 hours.
Let me know, sure, anything will help.
Can you provide both files (zipped) so I can check if there is a way to optimize the code for such scenario?
Sheen91
Regular Member
- Dec 2, 2008
- 216
- 111
Hi Sweet,
I bought scrapebox quite some time ago but since have slowed down on IM and use a mac. Recently bought a PC as well and was looking to collate useful software for IM. I was wondering what proof you need etc. Would you be able to PM/start conversation with me. Thanks
I bought scrapebox quite some time ago but since have slowed down on IM and use a mac. Recently bought a PC as well and was looking to collate useful software for IM. I was wondering what proof you need etc. Would you be able to PM/start conversation with me. Thanks
- May 10, 2012
- 9,541
- 7,560
ganesha7
Regular Member
- Dec 8, 2012
- 363
- 64
Thanks, we really appreciate it! If you want to help, rather than donate you could purchase a premium plugin or another SB license and it will help you plus help us keep SB running for the next 7 years.
hi, bought SB now.. transaction id starts with 0KF1... waiting for activation. thanks..
- Jan 25, 2009
- 6,378
- 3,774
ganesha7
Regular Member
- Dec 8, 2012
- 363
- 64
Pls check your pm.. thanks..
- Jan 25, 2009
- 6,378
- 3,774
I replied. Check you pm.
Sheen91
Regular Member
- Dec 2, 2008
- 216
- 111
Thanks mate.
- Status