1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What's the best way to scrape public proxies?

Discussion in 'Proxies' started by Monie, Jul 25, 2014.

  1. Monie

    Monie Regular Member

    Joined:
    Mar 22, 2014
    Messages:
    211
    Likes Received:
    42
    Okay, guys, I am breaking my head over this.

    So I just set up GSA SER on a VPS for the very first time in my life. Before kicking off my first link building project, I would of course need proxies. So I thought why not buy just 5 private proxies for posting and use public proxies for scraping? So I began hunting around for a good proxy scraper that would fit my needs - maybe I would even just start building links with just public proxies for now and get private proxies later.

    I tried two software - Proxy Multiply and Dexter Proxy.

    In spite of both of these software's big claims, both of them seem to work pretty poorly.

    Most proxies seem to stop working soon after they are scraped. For both of them I have to manually export the proxy list each time. But by the time I do export the proxy list and test it with Scrapebox, most of them have stopped working.

    I think No Hands Proxies can dynamically scrape new proxy lists continuously and periodically load them up into GSA automatically, although I haven't tried that yet. Am I right?

    Also, there are proxy list membership sites like IPFreelyProxies - I simply don't know how effective these are.

    Overall guys, I am super-confused. Is public proxies even an option, at least for scraping? Should I just buy 10 semi-dedicated proxies and just use them for everything from scraping to posting? Should I just forget about public proxies?
     
  2. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,295
    Likes Received:
    8,714
    tools that scrape public proxies will always have a problem as there are alot
    of tools on the market for scraping proxies, so all those tools all those people
    all scraping for same pool of proxies = fast death, as u have noticed.

    your options are as u say private proxies, posting
    dedicated for scraping,

    but as ive also said a thousand times, port scanned public proxies, proxies
    sourced from scanning networks, ip blocks, will last 10x longer than public proxies
    scraped, this is because anyone with a scraper can scrape proxies
    but a scraper cant scrape port scanned proxies even though there still public
    so there is a use, just gotta find right person

    ill just mosey along and whistle - public port scanned proxies

    [​IMG]
     
    Last edited: Jul 25, 2014
  3. Monie

    Monie Regular Member

    Joined:
    Mar 22, 2014
    Messages:
    211
    Likes Received:
    42
    Holy cow! :O What exactly can I get these port scanned public proxies, and how do I get them? Can you elaborate?

    Also how do you automatically load up these proxies into GSA, without having to manually do it?
     
    • Thanks Thanks x 1
  4. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,295
    Likes Received:
    8,714
  5. Creative9

    Creative9 Junior Member

    Joined:
    May 22, 2014
    Messages:
    136
    Likes Received:
    15
    Just use scrapebox.
     
  6. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,295
    Likes Received:
    8,714
    scrapebox is no different to the other tools hes using scraping public
    proxies that are dying fast for him
     
  7. jimbobo2779

    jimbobo2779 Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 17, 2008
    Messages:
    3,245
    Likes Received:
    2,397
    Occupation:
    Software Engineer
    Location:
    UK
    Home Page:
    Despite what proxygo states, not all proxy harvesters are the same, you may have noticed with other tools you get anywhere from 10-50 Google passed proxies working at any time if you are lucky. With No Hands Proxies (link in my sig) you will be getting more in the region of 100-500 (I just took a snapshot from my test server to see how many it was running with from NHP, current list is 342 working Google passed proxies that were tested no more than 12 minutes prior. The list is posted below for proof:

    Code:
    118.238.234.228:3128
    37.60.184.60:80
    193.36.34.13:3128
    112.175.69.132:3128
    190.254.1.202:8080
    88.212.28.130:3128
    86.125.59.121:3128
    201.62.64.120:3128
    79.127.127.169:3128
    79.127.126.220:3128
    85.25.144.24:8080
    189.6.244.27:3128
    186.190.238.65:8080
    189.39.124.54:80
    213.233.161.195:3128
    80.80.247.13:3128
    187.2.153.142:8080
    186.210.204.74:3128
    219.90.127.35:3128
    88.255.251.17:8080
    189.90.111.230:3128
    180.183.142.120:3128
    113.196.34.66:8080
    140.116.164.102:3128
    177.6.124.68:3128
    180.183.122.108:3128
    23.102.160.68:3128
    200.135.250.130:3128
    121.179.211.221:3128
    41.191.235.78:8080
    177.130.51.63:3128
    190.12.81.173:8080
    180.76.2.16:80
    190.207.207.240:8080
    123.30.40.53:3128
    200.86.217.74:3128
    220.227.50.164:3128
    190.39.243.228:8080
    186.91.253.102:8080
    190.198.164.185:8080
    190.203.41.235:8080
    190.206.179.252:8080
    79.134.4.241:3128
    201.211.89.161:8080
    1.173.113.27:3128
    186.88.103.3:8080
    186.94.52.190:8080
    190.204.87.128:8080
    190.205.222.178:8080
    190.207.182.38:8080
    190.207.34.166:8080
    201.242.86.245:8080
    190.201.192.207:8080
    62.48.214.96:3128
    186.89.122.126:8080
    186.90.44.53:8080
    186.91.160.62:8080
    186.91.35.243:8080
    186.93.97.109:8080
    186.93.97.254:8080
    186.95.12.37:8080
    186.95.204.194:8080
    190.198.221.251:8080
    190.201.184.120:8080
    190.203.111.229:8080
    190.203.196.254:8080
    190.206.177.157:8080
    190.207.62.228:8080
    190.37.75.67:8080
    190.72.25.68:8080
    190.73.45.175:8080
    190.78.112.41:8080
    200.84.248.83:8080
    201.242.53.36:8080
    186.90.112.33:8080
    186.92.102.214:8080
    186.93.16.136:8080
    186.94.60.92:8080
    186.95.218.62:8080
    190.200.38.187:8080
    190.73.121.183:8080
    190.200.25.122:8080
    193.172.141.219:80
    186.89.78.138:8080
    190.39.105.155:8080
    190.254.1.201:9090
    153.122.26.4:3128
    200.108.132.166:3128
    187.6.50.250:3128
    1.174.40.185:8088
    187.44.14.19:8080
    186.94.3.101:8080
    190.72.160.189:8080
    190.79.6.37:8080
    201.211.226.115:8080
    186.93.104.126:8080
    173.220.212.68:80
    190.207.223.71:8080
    200.84.96.20:8080
    190.202.245.43:8080
    190.206.53.102:8080
    201.211.252.69:8080
    190.78.19.119:8080
    190.77.208.84:8080
    190.78.113.56:8080
    186.95.19.92:8080
    201.211.239.238:8080
    201.248.114.100:8080
    201.211.189.73:8080
    190.203.112.52:8080
    190.206.80.79:8080
    89.248.118.232:3128
    200.205.140.114:3128
    190.37.69.36:8080
    125.24.76.177:8080
    190.207.31.30:8080
    186.89.154.129:8080
    201.210.90.37:8080
    190.202.255.150:8080
    190.74.76.167:8080
    190.39.70.100:8080
    200.109.52.197:8080
    186.89.210.73:8080
    125.24.78.9:8080
    23.92.222.117:7808
    23.226.131.35:80
    199.189.84.217:3128
    190.37.210.165:8080
    186.93.104.95:8080
    201.248.254.18:8080
    186.90.26.86:8080
    190.206.68.201:8080
    186.91.195.190:8080
    190.78.185.114:8080
    190.206.68.247:8080
    190.203.152.37:8080
    93.95.196.112:3128
    190.200.4.85:8080
    186.91.100.217:8080
    186.91.84.177:8080
    186.92.15.192:8080
    186.95.234.122:8080
    186.95.79.3:8080
    190.198.70.126:8080
    190.198.77.118:8080
    190.204.67.20:8080
    190.205.225.181:8080
    190.38.100.156:8080
    190.38.187.45:8080
    190.73.123.83:8080
    190.94.210.190:8080
    201.243.30.3:8080
    125.24.77.167:80
    125.24.78.193:8080
    125.24.78.202:80
    186.14.214.88:8080
    186.14.223.224:8080
    186.88.160.215:8080
    186.88.215.95:8080
    186.88.93.182:8080
    186.89.187.133:8080
    186.89.23.248:8080
    186.90.56.52:8080
    186.91.127.158:8080
    186.91.162.149:8080
    186.91.203.59:8080
    186.91.232.59:8080
    186.91.64.11:8080
    186.92.248.203:8080
    186.92.51.33:8080
    186.93.141.231:8080
    186.93.184.133:8080
    186.93.200.201:8080
    186.93.207.74:8080
    186.93.232.113:8080
    186.94.213.69:8080
    186.94.28.246:8080
    186.94.82.143:8080
    186.95.102.139:8080
    186.95.102.86:8080
    186.95.207.19:8080
    186.95.210.23:8080
    186.95.228.111:8080
    186.95.80.68:8080
    188.134.91.99:3128
    190.198.156.243:8080
    190.198.20.193:8080
    190.198.248.168:8080
    190.199.45.174:8080
    190.200.225.174:8080
    190.201.126.190:8080
    190.201.15.34:8080
    190.201.172.126:8080
    190.203.101.147:8080
    190.203.227.33:8080
    190.203.241.20:8080
    190.203.246.247:8080
    190.203.69.153:8080
    190.204.111.29:8080
    190.204.81.242:8080
    190.205.116.16:8080
    190.205.151.184:8080
    190.205.164.188:8080
    190.206.15.84:8080
    190.207.3.138:8080
    190.207.46.110:8080
    190.207.47.107:8080
    190.36.179.251:8080
    190.36.26.33:8080
    190.37.100.108:8080
    190.37.100.153:8080
    190.37.104.211:8080
    190.37.110.177:8080
    190.37.88.14:8080
    190.37.93.169:8080
    190.38.179.163:8080
    190.38.182.250:8080
    190.38.245.41:8080
    190.38.25.116:8080
    190.38.59.117:8080
    190.39.129.195:8080
    190.39.246.234:8080
    190.39.55.231:8080
    190.39.75.143:8080
    190.39.85.63:8080
    190.39.97.203:8080
    190.72.183.56:8080
    190.72.19.170:8080
    190.72.226.205:8080
    190.72.230.74:8080
    190.73.118.205:8080
    190.73.127.203:8080
    190.73.131.11:8080
    190.73.138.97:8080
    190.73.239.64:8080
    190.73.99.65:8080
    190.74.95.39:8080
    190.75.118.17:8080
    190.75.160.64:8080
    190.75.181.25:8080
    190.75.203.147:8080
    190.77.182.90:8080
    190.77.212.249:8080
    190.78.148.234:8080
    190.78.173.81:8080
    190.78.21.82:8080
    190.78.23.139:8080
    190.78.58.209:8080
    190.78.61.38:8080
    190.79.151.136:8080
    190.79.225.118:8080
    190.79.26.193:8080
    190.79.7.44:8080
    197.211.32.170:3128
    200.109.149.219:8080
    200.109.150.240:8080
    200.8.4.26:8080
    200.82.146.136:8080
    200.82.180.196:8080
    200.84.103.72:8080
    200.84.128.213:8080
    200.84.142.63:8080
    200.84.233.126:8080
    201.208.103.249:8080
    201.208.191.116:8080
    201.209.225.113:8080
    201.210.238.40:8080
    201.210.240.85:8080
    201.211.108.155:8080
    201.211.112.19:8080
    201.211.119.231:8080
    201.211.137.248:8080
    201.211.164.217:8080
    201.211.192.206:8080
    201.211.251.87:8080
    201.242.232.18:8080
    201.242.45.135:8080
    201.242.64.176:8080
    201.242.90.79:8080
    201.243.11.60:8080
    201.243.174.134:8080
    201.248.250.239:8080
    201.248.97.61:8080
    201.4.236.83:8080
    41.133.127.51:80
    54.179.135.50:3128
    88.80.188.161:3128
    91.103.91.144:8080
    91.137.149.19:3128
    201.209.126.126:8080
    190.199.250.233:8080
    190.74.63.1:8080
    186.93.66.199:8080
    37.32.24.67:8080
    36.74.72.205:3128
    190.203.150.3:8080
    190.79.5.212:8080
    190.72.47.252:8080
    186.93.230.206:8080
    190.203.153.137:8080
    190.200.54.21:8080
    212.232.52.56:8080
    190.94.236.161:8080
    190.207.32.182:8080
    190.206.164.165:8080
    186.90.154.164:8080
    201.209.195.64:8080
    186.89.60.200:8080
    186.91.61.118:8080
    186.92.1.218:8080
    46.175.184.235:8080
    186.88.231.182:8080
    186.94.203.210:8080
    190.204.67.229:8080
    190.36.1.108:8080
    201.211.165.103:8080
    190.36.16.253:8080
    201.242.168.5:8080
    31.43.21.227:3128
    185.30.147.197:8080
    186.90.48.139:8080
    190.92.62.254:3128
    190.74.168.149:8080
    186.89.72.192:8080
    190.73.226.100:8080
    89.28.73.163:8080
    186.93.217.183:8080
    190.79.227.54:8080
    201.210.232.191:8080
    190.200.209.194:8080
    190.206.24.69:8080
    190.39.252.46:8080
    179.185.22.83:3128
    118.70.177.90:8888
    180.234.210.50:80
    190.204.148.99:8080
    37.187.88.169:9876
    182.253.48.185:3128
    190.201.25.180:8080
    186.88.205.162:8080
    186.92.39.227:8080
    190.79.13.48:8080
    Obviously if you test these proxies then you will have to take into consideration the time between me posting and you testing but this is what you can expect from running with NHP. Yes some will go down but then more will be found so you will always be running with the freshest list possible.

    And no it doesn't just harvest from a pre-determined list of sites like a lot of checkers do, your list is unlikely to be the same as others' lists as it will search in Google to find more sources besides the ones built in. What separates it from all other harvesters is that it will re-check proxies on the fly whenever they get to a certain age (in my test server case it is whenever a proxy was checked more than 12minutes ago), I am not aware of any other tool that does this.

    I am not saying that proxygo's service is bad but for him to state that all tools do X or that you will be using the same bad proxies that hundreds of users are also using is a bit disingenuous as he knows full well that his proxy list is being used by lots of like minded users and is well open to the same abuse as any other list of public proxies. At least with NHP you can be sure the proxies are usable when you want to use them.
     
    Last edited: Jul 27, 2014