1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

ScrapeBox Optimal Connections Settings

Discussion in 'Black Hat SEO Tools' started by whitez, Jan 18, 2011.

  1. whitez

    whitez Newbie

    Joined:
    Oct 22, 2009
    Messages:
    44
    Likes Received:
    2
    Hi Guys,
    What is your maximum/optimal connection settings for ScrapeBox?

    Mine is:

    Internet connection: 9mb/s
    Url Harvester: 10 - Timeout: 30 sec
    Fast Poster: 50 - Timeout: 35 sec
    Link Checker: 35 - Timeout: 60 sec
    Proxy Harvester: 20 - Timeout: 15 sec
     
  2. jheyslow

    jheyslow Senior Member

    Joined:
    May 11, 2010
    Messages:
    876
    Likes Received:
    710
    Occupation:
    trying new things
    Internet connection: 30-60 mb/s
    Url Harvester: 30 - Timeout: 50 sec
    Fast Poster: 150 - Timeout: 90 sec
    Link Checker: 150 - Timeout: 90 sec
    Proxy Harvester: 50 - Timeout: 30 sec[/QUOTE]

    That is is the settings I used now.
     
    • Thanks Thanks x 1
  3. antsaoo

    antsaoo Supreme Member

    Joined:
    Oct 1, 2008
    Messages:
    1,291
    Likes Received:
    637
    Was wondering this and been trying optimize for last 8 hours lol.

    100mbs

    Harvester 350 per Engine works fine

    But the link checker hangs no matter how less threads i put in . Then it goes really fast for little time and slows down again. Without proxies. This is killing me as im trying to think how to get it working better. Win server 2003.

    Fast Poster 100 threads - 90s timeout
     
  4. jheyslow

    jheyslow Senior Member

    Joined:
    May 11, 2010
    Messages:
    876
    Likes Received:
    710
    Occupation:
    trying new things

    Try using free proxies.
     
  5. antsaoo

    antsaoo Supreme Member

    Joined:
    Oct 1, 2008
    Messages:
    1,291
    Likes Received:
    637
    Umm i don't see any point using proxies if checking links. :/ Am i missing something ? :p
     
  6. dgfalk

    dgfalk Power Member

    Joined:
    Apr 26, 2010
    Messages:
    687
    Likes Received:
    94
    I am wondering the same as well. I just bought 5 private proxies today and pretty much only plan on using fast post, I have a 10mbps connection. What would you guys recommend for connections and timeout?
     
  7. jheyslow

    jheyslow Senior Member

    Joined:
    May 11, 2010
    Messages:
    876
    Likes Received:
    710
    Occupation:
    trying new things

    Ops I thought it needs proxy, Try asking support about it.
     
  8. loopline

    loopline Jr. VIP Jr. VIP

    Joined:
    Jan 25, 2009
    Messages:
    3,374
    Likes Received:
    1,799
    Gender:
    Male
    Home Page:
    Dedi Servers Internet connection: 1Gbit/s

    Url Harvester: depends on engine and proxies.

    Yahoo -10 connections with no proxies
    Google and AOL - Use private proxies at 20% of my proxies. So if I use 100 private proxies I set connections to 20. Can harvest for quite a while like this, and private proxies are so fast, I find my urls/s come out more then more public proxies, plus IMHO you don't have to mess around with failed ones, as 99.9% of them succeed first time with private proxies

    Fast Poster: 500 - 60 sec timeout

    Link Checker: 500 - 90 sec timeout on small runs, lower connections seem to be faster on my server when the list goes over 400K

    Proxy Harvester: 100 - 15 sec timeout.


    ~~~~~~

    When I used 100Mbit dedi servers

    Link checker 100 - 90 sec timeout
    Fast Poster - 200 - 50 sec timeout
    Proxy Harvester - 100 - 15 sec timeout
    Url Harvester - 75 per engine, I only ran 1 engine per scrapebox instance.


    ~~~~~~

    Home internet - 18Mbit

    Link checker 100 - 90 sec timeout
    Fast Poster - 100 - 50 sec timeout
    Proxy Harvester - 100 - 15 sec timeout
    Url Harvester - 75 per engine, I only ran 1 engine per scrapebox instance.



    I didn't see much difference in many settings as far as optimal from my home 18Mbit connection to my server 100Mbit except that the server just did everything faster. When I jacked connections success seemed to fall.

    When I upgraded to my 1 Gigabit I am able to max everything and it sails smooth as can be.




    What I do is start low, and turn it up until speed and results tend to fall off rather then increase and thats about where you need to be. Of course its subjective, cause the larger the file you work with, the more the processor has to compensate with, even if there is enough RAM, and of course if there is not enough RAM then you have issues.

    This happens to me even on my 1Gbit server when checking links at 500 connections. So long as your not getting lots of timeout errors then its fine. Basically what happens is you have a lot going on there. Windows uses a message Que to tell the scrapebox window to repaint (refresh what your viewing) and when you have tons happening your pc may not have enough resources to do everything Windows and scrapebox and whatever else you have running, are asking it to do. So it can't process those messages fast enough and the screen "hangs" or ghosts or even says not responding. Even my 6 core servers with 8GB of ram on 1Gbit lines do this and sometimes even for 10 mins at a go. I have learned don't touch it, the pc is doing what it needs, let it be. (or buy a faster pc for instant gratification lol - I spend too much as it is).

    If you are getting lots of timeout errors, then your inet connection pipe isn't big enough to handle it and it times out before it has a chance to load the page. When you have 100 pages that all time out at a time it will seem "jerky" in motion as it skips. In which case just turn your connections down/timeouts up.

    So basically not a lot of timeout errors, leave it be. Lots of timeout errors, then turn your connections down and timeouts up.

    Oh, and no you don't need proxies when link checking, although its good to use the randomize setting so you aren't link checking 300 pages on 1 domain at a time.
     
    • Thanks Thanks x 8
    Last edited: Jan 18, 2011
  9. antsaoo

    antsaoo Supreme Member

    Joined:
    Oct 1, 2008
    Messages:
    1,291
    Likes Received:
    637
    Ye but the problem kinda is its not taking even 10% of processor or memory. Guess its the connection opening / closing.

    Also to mention i noticed funny glitch while scraping. When you start harvester. Open up view changelog for example while scraping and i got to 2k/s with google 2.5k urls/s with yahoo. 2s public proxies.Dunno if its only some error on my dedi or it fucks up something. Though i tried on vps and it maxed to 10mbs at all times too :O ( mofos did not tell it was limited at their site =/)
     
  10. smartgirl

    smartgirl Regular Member

    Joined:
    Nov 15, 2008
    Messages:
    274
    Likes Received:
    75
    I have a Toshiba laptop, Intel Core 2 Duo, CPU T7500, 2.20 GHz,RAM 2 MB , wireless connection 100Mbit. For fast poster I set 50 connections, 90 timeout and I get around 100 links processed per minute which I find very low as my lists are quite big.

    What can be done to increase the speed of links processing? A better home computer, private proxies, etc?

    Thanks!
     
  11. general_z

    general_z Junior Member

    Joined:
    Aug 12, 2008
    Messages:
    120
    Likes Received:
    32
    Internet connection: 10mb/s, full duplex
    Url Harvester: 30 - Timeout: 10 sec
    Fast Poster: 150 - Timeout: 40 sec (5-7% more success @ 30 connections, 60s timeout, but 4x the time)
    Link Checker: 50 - Timeout: 40 sec
    Proxy Harvester: 50 - Timeout: 3 sec

    Comp
    AMD x6 1055T @ 4ghz
    8gb ram
    Win7 64 Ultimate
     
  12. general_z

    general_z Junior Member

    Joined:
    Aug 12, 2008
    Messages:
    120
    Likes Received:
    32
    You can edit it yourself at YOUR-SCRAPEBOX-DIR/scrapebox.ini
     
  13. ningning

    ningning Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 22, 2010
    Messages:
    925
    Likes Received:
    260
    I have a pretty similar setup with 10 private proxies.

    I can get ~200 per minute. But the then failure rate is 50%...

    Any suggestions?
     
  14. webwowster

    webwowster Newbie

    Joined:
    May 17, 2009
    Messages:
    6
    Likes Received:
    0
    When you say 100mb's do u really mean 1mb = 1000kbps?. I am on the fastest speed adsl2 through internode and when i did a speed test i am only getting 3872 kbps so in other words nearly 4mbs?? :confused: I am running on 3ghz system with 2gb ram. No idea what to put my settings at!
     
  15. jharr53301

    jharr53301 Regular Member

    Joined:
    Apr 10, 2011
    Messages:
    446
    Likes Received:
    42
    Occupation:
    eBay Power Seller and Internet Sales and Marketing
    Location:
    The Burgh
    This is very helpful info thanks alot
     
  16. bstsystems

    bstsystems Regular Member

    Joined:
    May 17, 2011
    Messages:
    465
    Likes Received:
    81
    How would I tell if I have the url harvester connections set too high, would it just not work or what?
     
  17. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    111
    What do you all use for pagerank?
     
  18. kokoloko75

    kokoloko75 Elite Member

    Joined:
    Jan 1, 2011
    Messages:
    1,628
    Likes Received:
    1,935
    Occupation:
    Design director
    Location:
    Paris (France)
    For Proxy Checker, I use 200 connections with 2x Re-check Failed.
    Proxygo said 50 connections is the best choice for the proxy checker, to get real results.

    Beny

    EDIT : So old thread ! Wtf to bump this ?!
     
    Last edited: May 15, 2012