1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Linux CLI proxy checker

Discussion in 'Proxies' started by Domo_18895, Jul 22, 2013.

Thread Status:
Not open for further replies.
  1. Domo_18895

    Domo_18895 Newbie

    Joined:
    Jan 29, 2013
    Messages:
    34
    Likes Received:
    11
    Anyone know of a good proxy checker that can be used from the command line in Linux? I don't need to scrape proxies just test the list that I have.
     
  2. cnick79

    cnick79 Jr. VIP Jr. VIP

    Joined:
    Jun 10, 2010
    Messages:
    653
    Likes Received:
    341
    Location:
    Google's SandBox
    You can do something like this:

    You can try to get a webpage using the proxy and if that fails, the proxy is no good. You could put this in a script too if you wanted to...

    EDIT: You might also try using the nc command.
     
    Last edited: Jul 22, 2013
  3. Domo_18895

    Domo_18895 Newbie

    Joined:
    Jan 29, 2013
    Messages:
    34
    Likes Received:
    11
    Cool wasn't even thinking of wget. Is this simple to multiprocess (ie: just spawn out tons of these at once)?
     
    Last edited: Jul 22, 2013
  4. cyrix

    cyrix Junior Member

    Joined:
    Sep 19, 2008
    Messages:
    179
    Likes Received:
    61
    Occupation:
    Full Time Internet Marketer\Developer
    Location:
    United States
    Here is a multi-threaded proxy checker written in perl I found from a quick google search. Try it out.

    Code:
    #!/usr/bin/perl -w
    # Proxy list tester using threads
    #
    # INPUT file -> "proxy"
    # format -> IP:PORT on each row
    #
    # OUTPUT file -> "proxy.log"
    
    use threads;
    use WWW::Mechanize;
    
    $nn = 25; # number of simultaneous connections
    $timeout = 10; # proxy request timeout (secs)
    
    sub start_thread {
            my @args = @_;
    	my $go = WWW::Mechanize->new( agent=> "Mozilla/5.0", timeout => $timeout);
    	$go->proxy(['http'], 'http://'.$args[0].'/');
    	print "Testing: ".$args[0]."\n";
    	eval {
     	 $go->get('http://automation.whatismyip.com/n09230945.asp');
    	};
    	if ($@) { return; }
    	$match = $go->content;
    	my($crap,$ip)=split(/^(.*):/,$args[0]);
    	if ($match =~ /$ip/) {
    		open(LOG,">> proxy.log"); print LOG $args[0]."\n"; close(LOG);
    		print "======> UP $args[0] <=======\n";
    	}
    }
    
    @proxy=`cat proxy`;
    $n = 0;
    foreach $i (@proxy) {
    chomp($i);
    $thr[$n] = threads->create('start_thread', $i);
    $n++;
    if ($n == $nn) {
     `sleep $timeout`;
     for ($x=0; $x<$nn;$x++) {
      $thr[$x]->join();
     }
     $n = 0;
    }
    
    }
    
     
  5. intellex

    intellex Newbie

    Joined:
    Dec 29, 2009
    Messages:
    23
    Likes Received:
    0
    Simply grabbing something found from a "quick" Google search or off of stackoverflow and then using it blindly can royally mess things up for you if you don't analyze what you're about to use.

    As the case in the previous message, that script is really poorly coded. Yes it'll do the job for you, but say if you got 10k proxies, you'll eventually be inefficiently using system resources because this script doesn't detach threads and doesn't free up memory. So it'll continue to add threads without freeing up the finished threads and then it'll use up all your ram with useless finished threads.

    I'll post a better script once I'm done testing the one I'm working on right now.
     
    Last edited: Feb 8, 2014
  6. gacott

    gacott Junior Member

    Joined:
    Mar 10, 2009
    Messages:
    153
    Likes Received:
    62
    Wow, still working on that script hu?
     
    • Thanks Thanks x 1
  7. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,203
    Likes Received:
    8,689
    and your comment added what to this thread exactly
    other than to bump a thread from the start of the year to add a
    pointless comment
     
  8. gacott

    gacott Junior Member

    Joined:
    Mar 10, 2009
    Messages:
    153
    Likes Received:
    62
    Hmm, the comment did exactly what you stated, it bummed the thread. Why? Because I hope that he smacks his forehead, says "oh yeah, I forgot" and comes here to post his script. Thereby adding value and not pointless.

    Thanks
     
  9. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,203
    Likes Received:
    8,689
    dont u think after 13 months if he was going to post it,
    he would have done so already, 13 months is along time
    considering he logs into the forum daily. u could have just pmd him
     
Thread Status:
Not open for further replies.