My scrapebox timeout at 100 Mb/s internet connection

[alaltaierii]

So, I have 100 Mb/s internet connection but I can't use scrapebox at full speed

For example,when I run blog analyzer with more than 50 connections, after about 1 minute from starting I receive only "error time out" and error 404 blogs. I also tried with 250 connections and the same thing happened. I saw how the blogs are correctly analysed and how netlimiter shows me the incoming speed (between 1000 and 2500 kb/s). But after about 1-2 minutes..boom...netlimiter shows 0 kb/s and I receive only time out errors or the blog analyzer window freeze. Even if for big lists I split them in 50k parts.

Same thing with the proxy harvester. At 40 connections, a lot of time out errors even with 60 seconds time out.

I'm using public proxies with under 2000ms filter (about 80-100 proxies).

My notebook is a core 2 duo t6600 ,4 gb ram so I don't think this can be the problem for freezing or time out errors.

The thing is I don't receive the internet directly in my notebook. This machine it's in a network with my main computer. Can be this the problem ? Do I need to make some special TCP/IP settings or something like that ?



ps: Can I run blog analyzer without proxies and without have problems with my ISP ?

waiting for your answers guys

thanks

 

[Paranoid Android]

check your proxies man, maybe its a proxy timeout. or maybe you could get a router and try to connect your lappy directly to the modem instead of through your computer.

 

[alaltaierii]

What do I need to check ? there are public proxies harvested.tested and rechecked many times before I run blog analyzer or other addons. All are filtered under 2000ms.
But I only used about 100 proxies. This can be a problem if I use many connections ?

I will try to connect my notebook with a router but till then I want to use scrapebox at full capacity

 

[semilar]

• Proxies can be checked with sb's proxy checker.
• What OS do you use? XP may need a half open connections patch. 7 is fine. Not sure about Vista.
• What antivirus software do you use? I used avg free which (like most modern av's) has a module that watches internet traffic, which is a must btw for sb. But in AVG's case it did not work well, led to timeouts and had some sort of data leak that was filling up my ram. AGV sucks. If so, try avast.
• Check the rest of your Network configuration. Does your modem / router have problems with a lot of parallel connections? Maybe you need better hardware.
• Do NOT use WLAN to connect your pc to your router, that technology is not 100% reliable, esp when using tools like torrent or scrapebox. Use a good old lan cable.
• And btw, read what the different error codes mean - a 404 means page not found, it's no timeout.
  However, its possible that a faulty proxy gives out a 404 for every site you try to view through it.
• And even with working proxies and a perfect setup pc and fast connection you will get some timeouts from slow/not responding websites, but thats mostly their fault then.

So check all of these factors and then play with the # of connections and # timeout settings in scrapebox. The bandwith meter plugin might come handy too to see if you already max out your connection.
To have a reference for your TRUE bandwith, first do some online bandwith speed tests. Maybe your connection is labeled 100mbit but in reality only supports 20mbit (This is i.e. true for DSL connections, where the companies sell you a theoretical speed which can be only reached if only a few of your neighbours use dsl too and you are not far away from the next switch, similar technical limitations might apply for other connection technologies)
So do the bandwith tests and try to adjust scrapebox to stay below the measured speed.

Hope that helps.

 

[alaltaierii]

• What OS do you use? XP may need a half open connections patch. 7 is fine. Not sure about Vista.
  
  I use the patch and set it up to 100
  
  
• What antivirus software do you use? I used avg free which (like most modern av's) has a module that watches internet traffic, which is a must btw for sb. But in AVG's case it did not work well, led to timeouts and had some sort of data leak that was filling up my ram. AGV sucks. If so, try avast.
  
  I have avast on my main PC and also on my notebook. Do I need to set something ?
  
  
• Check the rest of your Network configuration. Does your modem / router have problems with a lot of parallel connections? Maybe you need better hardware.
  
  
  I don't use a modem/router. Direct cable connection with Internet sharing (I have 2 network cards on my main PC)
  
  
• And btw, read what the different error codes mean - a 404 means page not found, it's no timeout.
  However, its possible that a faulty proxy gives out a 404 for every site you try to view through it.
  
  It give me 404 for the same reason. Because I use fresh proxies and the site is working if I acces it manually.
  
  
• And even with working proxies and a perfect setup pc and fast connection you will get some timeouts from slow/not responding websites, but thats mostly their fault then.
  
  yes I know but a lower rate of timeouts or not responding website
  

So check all of these factors and then play with the # of connections and # timeout settings in scrapebox. The bandwith meter plugin might come handy too to see if you already max out your connection.
To have a reference for your TRUE bandwith, first do some online bandwith speed tests. Maybe your connection is labeled 100mbit but in reality only supports 20mbit (This is i.e. true for DSL connections, where the companies sell you a theoretical speed which can be only reached if only a few of your neighbours use dsl too and you are not far away from the next switch, similar technical limitations might apply for other connection technologies)
So do the bandwith tests and try to adjust scrapebox to stay below the measured speed.

I made the bandwith tests. Is not 100 Mb/s but is 70-80 Mb/s


Hope that helps.

I connected the main internet cable to my laptop and I tested Scrapebox. It's working very good !

So, I guess is something from my network. And I don't know what can be

 

[semilar]

So you had your modem connected to your pc, and your notebook to the second lanport on your pc with internet connection sharing?
Could be quite a few reasons there why it does not work right, my tip, get a high quality router w. integrated modem. check first with the producer and in internet forums if the router is up to a high connection load task.

Standard avast settings are fine for me.

404 - must be the proxies then. See, you only get a 404 once you successfully connected to the website. Only exception is when you are using a proxy, it by itself can show you a 404 too. So the most likely explanation for the 404's is unreliable proxies if the website can be reached directly.

Only thing you can do is retry commenting over and over (because scrapebox should use different proxies with every try) to get the highest success rate and try to recheck your actual proxy list prior to that.

But then again you say you get less 404 w/o internet sharing? Hmm. Thats strange. I guess the best thing to do is get a high quality router.

Anyways, glad to see that its working now.

 

[alaltaierii]

So you had your modem connected to your pc, and your notebook to the second lanport on your pc with internet connection sharing?
Could be quite a few reasons there why it does not work right, my tip, get a high quality router w. integrated modem. check first with the producer and in internet forums if the router is up to a high connection load task.

Standard avast settings are fine for me.

404 - must be the proxies then. See, you only get a 404 once you successfully connected to the website. Only exception is when you are using a proxy, it by itself can show you a 404 too. So the most likely explanation for the 404's is unreliable proxies if the website can be reached directly.

Only thing you can do is retry commenting over and over (because scrapebox should use different proxies with every try) to get the highest success rate and try to recheck your actual proxy list prior to that.

But then again you say you get less 404 w/o internet sharing? Hmm. Thats strange. I guess the best thing to do is get a high quality router.

Anyways, glad to see that its working now.

yes, to the first question. this is how I get internet on my notebook.
Well, I'm not so glad because I cannot use the internet only for my notebook(the only way that scrapebox works now). I need to share it with the main PC.
Anyway...I will check out for a quality router

thank you for your answers

 

[semilar]

Internet connection sharing was a bad solution in any case, gave up on that one loooong ago... believe me, a good router makes your life easier. Plus, some routers have nice features such as using usb drives as network drives, usb printer sharing over the network etc... I love that stuff. Don't forget to hit the thanks button

 

[BSON]

Why you are using proxies for blog analyzer?

 

[nipester]

So, I have 100 Mb/s internet connection

First things first. Which little birdie told you that?

edit: Seems like you fixed it so not sure if the question is still valid.

 

[redstone.1337]

I don't think you'd need proxies for blog analyzer as you are just visiting those blogs for the analysis.

 

[alaltaierii]

First things first. Which little birdie told you that?

edit: Seems like you fixed it so not sure if the question is still valid.

I can get more than 75 Mb/s if I close utorrent and other programs.

Why you are using proxies for blog analyzer?

I don't know I just don't want to have any problem with my ISP. Can I analyze without proxies without any risk ?

 

[xboxps3wow]

i don't know whats wrong, but its not your connection. I'm using a ~3mbs down and 2mbs up for scrapebox and it works perfectly.

 

[Anubis1980]

I can get more than 75 Mb/s if I close utorrent and other programs.

I don't know I just don't want to have any problem with my ISP.

75mb/s ? good one :fing26:

Can I analyze without proxies without any risk ?

yes. I use it to test my lists.No risk. If you use firefox with tons
of plugins, many of them will connect to ips to get alexa rank,
pagerank etc pp..You could easily have 20 and more
connections open.

 

[hexa714]

I was having the same problem with >25 connections.... it was my router.

 

[andreyg13]

I don't think you'd need proxies for blog analyzer as you are just visiting those blogs for the analysis.

redstone is right, you don't need proxies for checking blogs, just use the BA without proxies, also try to use direct connection without router

 

[semilar]

When doing a bandwith test one should do it with all other traffic producing programs closed to produce accurate results.

Also, torrent, depending on your connection settings can be a major interference while using sb since both can be used in a way that really pushes the hardware to its maximum. Same for other p2p software.

I've heard from some people that their ISP's caused them trouble when using sb without proxies even if it's "only" for checking tens or hundreds of thousands of blogs in a very short period of time. I guess it depends on the isp you are using, I never heard any complain from mine and I used proxies only when sb was scraping from multiple pages from one host (i.e. google) and for posting... but I thought I'd better mention it.

P.S.: I wish I had a 100mbit connection too.

 

[JCena]

redstone is right, you don't need proxies for checking blogs, just use the BA without proxies, also try to use direct connection without router

You mean connection which is "Always on'' ??

 

[alaltaierii]

redstone is right, you don't need proxies for checking blogs, just use the BA without proxies, also try to use direct connection without router

I already said that when I used direct connection I don't have a problem. And I don't use a router. I'm using internet sharing from my main PC. But I will probably buy a router this days