[PERL] Script for Proxy Scraping many sources

dreadpixel

Banned - Selling outside marketplace
Joined
Jul 27, 2010
Messages
2,502
Reaction score
1,179
The code is simple and uses basic Perl modules to crawl the given pages from a file, and prints out a list of proxies:
Code:
#!/usr/bin/perl
# Usage: proxyScraper.pl [proxy_file]
use strict;
use warnings;
use WWW::Mechanize;
use Try::Tiny;
my $source_file=shift;
open (INPUT_FILE, "<$source_file")  || die "Can't open $source_file: $!\n";
my @sources = ;
my $crawler = WWW::Mechanize->new();
foreach (@sources) {
    try {
        $crawler->get($_);
        # hunt for IP:PORT combination
        my @ips= $crawler->text() =~ /(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\:\d{1,5})/g;
        foreach (@ips){
            print "$_\n";
        }
    } catch {
        warn "[!] Error, who cares\n";
    }
}

And you load your proxy sources like this:
http://aliveproxy.com/fastest-proxies/
http://multiproxy.org/anon_proxy.htm
http://www.ultraproxies.com/high-anonymous.html
http://www.atomintersoft.com/high_anonymity_elite_proxy_list
http://www.proxylists.net/
http://www.ip-adress.com/proxy_list/
http://www.dailiip.com/
http://www.samair.ru/proxy/ip-address-01.htm
 
How to save the proxy into txt file?

You open write, then write the proxy into the file, after you finish the loop, close the file.

Code:
text_file = open("Output.txt", "w")
text_file.write(proxyvariable)
text_file.close()
 
hey can you please update your script with the output option ? or do you have maybe a new version already?

thank you
 
hey can you please update your script with the output option ? or do you have maybe a new version already?

thank you

Sorry I dont have the time for this. Need to work out some improvements and bug fixing in my BST.
 
Back
Top