I wrote a proxy bot during Xmas. It is running on a high end 4 CPU box as many daemons. It scrapes the top 300 serp pages on google, bing, yahoo over an hour as well as hits 20 daily proxy list sites and the hardcoded scapebox proxies lists 4 times a day accumlating open proxies. It has 60 processes that are constantly checking the found proxy databases for updating.
So far since xmas it has discovered and tried 170,000+ proxies, of these it found 30,000+ of them active. As I write this message, it currrently has 1,588 proxies that are fast and alive. Working proxies found are removed due to being becoing unavailable or not responding back to a test loop within 3 seconds (I don't want slow overloaded proxies).
It also scans the headers in the loop test for any reference to the requesting ip address, requesting host name or any _X_ parameters or VIA parameters. It has a dump proxy list feature for scapebox and since it checks scrapeboxes hardcoded proxies sites, I only have to load and check this one proxy list. The list also works with firefox's premium proxy switcher plugin.
I will be adding:
. check entire class C range of found proxies
. check against spam stopper engines
What I am asking the bhw community for is:
. known spam stopper sites for the bot to check against, if anyone cares to share
. more daily proxy listing sites for the bot to scan, if anyone cares to share
. more features people would like to see
Thank you for your time! I and hoping to get the bot out of alpha and into beta with the next week
So far since xmas it has discovered and tried 170,000+ proxies, of these it found 30,000+ of them active. As I write this message, it currrently has 1,588 proxies that are fast and alive. Working proxies found are removed due to being becoing unavailable or not responding back to a test loop within 3 seconds (I don't want slow overloaded proxies).
It also scans the headers in the loop test for any reference to the requesting ip address, requesting host name or any _X_ parameters or VIA parameters. It has a dump proxy list feature for scapebox and since it checks scrapeboxes hardcoded proxies sites, I only have to load and check this one proxy list. The list also works with firefox's premium proxy switcher plugin.
I will be adding:
. check entire class C range of found proxies
. check against spam stopper engines
What I am asking the bhw community for is:
. known spam stopper sites for the bot to check against, if anyone cares to share
. more daily proxy listing sites for the bot to scan, if anyone cares to share
. more features people would like to see
Thank you for your time! I and hoping to get the bot out of alpha and into beta with the next week