jkwilson78
Regular Member
- Jun 24, 2010
- 224
- 313
I came across a site called liveipmap.com and was wondering if there was a way to use it to harvest sources of public proxies?
Essentially, they post a daily list of proxies they detect through what seems like a massive ip and port scanning effort.
The catch is that while they provide a list of ip addresses each day they do not provide the port number for the proxy.
So to be useful, you would need to scrape the site to grab their IP list, then scan the ports for those ips to find which port to use then check if the proxies are still useable through something like SB.
I had a scraper built to grab the proxy ip addresses they post and then I tried running those addresses through a port scanner to find the open ports but my knowledge of this is pretty weak and I pretty much came up empty handed.
Another issue is that you don't know how long ago they found the ip addresses they post. Just because they were posted a few hours ago doesn't mean they weren't discovered a week ago and are already dead.
On the site they mention a service called ip2proxy and with a little Google searching I found out it is a service that provides people with a list of daily proxies with the intent of blocking access to people using those IPs
I looked into this as well, figuring I could just pay for the service but once again they do not provide the port numbers for the ip so came up short again.
The thing is, their daily list of proxies is usually in the 80K plus range so it sure would be nice to tap into this source!
So....any ideas?
Essentially, they post a daily list of proxies they detect through what seems like a massive ip and port scanning effort.
The catch is that while they provide a list of ip addresses each day they do not provide the port number for the proxy.
So to be useful, you would need to scrape the site to grab their IP list, then scan the ports for those ips to find which port to use then check if the proxies are still useable through something like SB.
I had a scraper built to grab the proxy ip addresses they post and then I tried running those addresses through a port scanner to find the open ports but my knowledge of this is pretty weak and I pretty much came up empty handed.
Another issue is that you don't know how long ago they found the ip addresses they post. Just because they were posted a few hours ago doesn't mean they weren't discovered a week ago and are already dead.
On the site they mention a service called ip2proxy and with a little Google searching I found out it is a service that provides people with a list of daily proxies with the intent of blocking access to people using those IPs
I looked into this as well, figuring I could just pay for the service but once again they do not provide the port numbers for the ip so came up short again.
The thing is, their daily list of proxies is usually in the 80K plus range so it sure would be nice to tap into this source!
So....any ideas?
