Domo_18895
Newbie
- Jan 29, 2013
- 33
- 11
Anyone know of a good proxy checker that can be used from the command line in Linux? I don't need to scrape proxies just test the list that I have.
Linux CLI proxy checker
- Thread starter Domo_18895
- Start date
-
- Tags
- checker cli command line linux proxy
- Status
- Jun 10, 2010
- 2,400
- 1,676
Domo_18895
Newbie
- Jan 29, 2013
- 33
- 11
Cool wasn't even thinking of wget. Is this simple to multiprocess (ie: just spawn out tons of these at once)?
Last edited:
Here is a multi-threaded proxy checker written in perl I found from a quick google search. Try it out.
Code:
#!/usr/bin/perl -w
# Proxy list tester using threads
#
# INPUT file -> "proxy"
# format -> IP:PORT on each row
#
# OUTPUT file -> "proxy.log"
use threads;
use WWW::Mechanize;
$nn = 25; # number of simultaneous connections
$timeout = 10; # proxy request timeout (secs)
sub start_thread {
my @args = @_;
my $go = WWW::Mechanize->new( agent=> "Mozilla/5.0", timeout => $timeout);
$go->proxy(['http'], 'http://'.$args[0].'/');
print "Testing: ".$args[0]."\n";
eval {
$go->get('http://automation.whatismyip.com/n09230945.asp');
};
if ($@) { return; }
$match = $go->content;
my($crap,$ip)=split(/^(.*):/,$args[0]);
if ($match =~ /$ip/) {
open(LOG,">> proxy.log"); print LOG $args[0]."\n"; close(LOG);
print "======> UP $args[0] <=======\n";
}
}
@proxy=`cat proxy`;
$n = 0;
foreach $i (@proxy) {
chomp($i);
$thr[$n] = threads->create('start_thread', $i);
$n++;
if ($n == $nn) {
`sleep $timeout`;
for ($x=0; $x<$nn;$x++) {
$thr[$x]->join();
}
$n = 0;
}
}Simply grabbing something found from a "quick" Google search or off of stackoverflow and then using it blindly can royally mess things up for you if you don't analyze what you're about to use.
As the case in the previous message, that script is really poorly coded. Yes it'll do the job for you, but say if you got 10k proxies, you'll eventually be inefficiently using system resources because this script doesn't detach threads and doesn't free up memory. So it'll continue to add threads without freeing up the finished threads and then it'll use up all your ram with useless finished threads.
I'll post a better script once I'm done testing the one I'm working on right now.
As the case in the previous message, that script is really poorly coded. Yes it'll do the job for you, but say if you got 10k proxies, you'll eventually be inefficiently using system resources because this script doesn't detach threads and doesn't free up memory. So it'll continue to add threads without freeing up the finished threads and then it'll use up all your ram with useless finished threads.
I'll post a better script once I'm done testing the one I'm working on right now.
Last edited:
Wow, still working on that script hu?
- Nov 2, 2008
- 59,070
- 26,604
and your comment added what to this thread exactly
other than to bump a thread from the start of the year to add a
pointless comment
- Status