Does anybody else have this problem when scraping/checking proxies?

X

Elite Member
Jr. Executive VIP
Jr. VIP
Joined
Jun 30, 2007
Messages
1,725
Reaction score
2,023
So I have a few VPS's to run my tools on, but I also have a badass gaming desktop that I like to let run constantly.

But I have this problem (on the desktop computer):

When I run proxy scraping and checking tools, such as ZennoPoster's Proxy Checker, ProxyGoblin, and the built-in proxy tool for Search Engine Ranker, after a while my internet locks up and I can't do anything with it.

I can't navigate to websites, use my tools, or anything. (Google and other sites fail to load, Search Engine Ranker starts saying "unable to download" for every site it finds, etc.)

I only use one tool at a time, and only run about 40 threads or so... I'm on a 15mbps internet connection, so I feel that it shouldn't be using up ALL of it.

I've used TCPOptimizer and tried multiple settings, including "Optimal," and it doesn't seem to help.

Does anyone else have this problem? Anyone know a solution?

Thanks!
 
im guessing your maxing out your connection
if pages dont load - thats usually a sign.

i have 10 meg connection and if i set my scrapers
on to many cons pages wont load.
but the thing is, i dont scrape for proxies anymore

i port scan using 3000 connections and never have
an issue collecting public proxies that way + they last
longer less used.
 
Last edited:
  • Like
Reactions: X
cool thanks for the reply!

any suggestions for good port scanning tools to find proxies?
 
i use proxyfire to pull proxies from my ip range lists
i use a single ip port scanner to draw the proxies out
which proxyfire converts to ip lists

the trick is, knowing where to scan and on what ports
u wont learn that in 5 minutes or 5 months, it takes
days and weeks just to find a few good sources .
 
  • Like
Reactions: X
Diversify your proxy source (proxy forums etc) and tweak your harvest settings (connections and timeout) until you find a sweet spot.
 
  • Like
Reactions: X
Cut your connections in half.

I normally run about 15-20 on a similar line, when I go higher then I start seeing some false failures.
 
  • Like
Reactions: X
prob just drop cons a bit
 
  • Like
Reactions: X
interesting, thanks for your input everyone. I'll start with 20 connections and see how that works, it just seems weird that I can't run a few more simultaneously on this connection speed. Oh well, when I move to my new apt soon I'll be investing in wideband, that should help out some haha. Also gonna try to figure out how to pull off what proxygo said.

Thanks again everyone!
 
Last edited:
Back
Top