1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Best Type of Proxies for Scraping (specifics)

Discussion in 'Proxies' started by balkhat, Oct 18, 2013.

  1. balkhat

    balkhat Registered Member

    Joined:
    Sep 30, 2012
    Messages:
    77
    Likes Received:
    14
    I am wanting to know a little bit more detail in regards to the best types of proxies I should be using for scraping SE's (with Hrefer, so not only google)

    For harvesting, I realize that public proxies are the way to go when dealing with mass. That is pretty common knowledge. But I want to know more.

    What else should I be paying most attention to, and what should I not care about?

    - Some say 'Anonymous' is not super important (confusing).
    - Some say both HTTP & SOCKS work, SOCKS are faster, but harder to find.
    - Some say to stay away from China/Pakistan proxies (as they wont harvest)

    I realize that I can just do some tests and probably figure this stuff out on my own, and I am, but it would be super appreciated if somebody very knowledgeable about proxies could chime in and help me shortcut my testing ;)
     
  2. MikePNV

    MikePNV Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 7, 2013
    Messages:
    1,123
    Likes Received:
    106
    Location:
    www.Proxy-N-Vpn.com
    Home Page:
    You can use free public proxy but if you want to get a much better speed you must use private proxy. Also you will know that you are always anonymous and you will have a better experience.
    Yes, you can use SOCKS proxies.
     
  3. balkhat

    balkhat Registered Member

    Joined:
    Sep 30, 2012
    Messages:
    77
    Likes Received:
    14
    I didn't put enough emphasis on the 'harvesting MASS links' part.
     
  4. akacash

    akacash Jr. VIP Jr. VIP

    Joined:
    Jan 16, 2010
    Messages:
    806
    Likes Received:
    575
    Location:
    The Beach, USA
    A lot of this depends on what you're using them for. That will determine what anonimity level you'll need, and that will give you some direction as to what to look for. If L3's(Transparent) proxies will do the job then there's thousands to choose from. If you need to completely hide your IP that will bring less results. For something to start with for bulk links to harvest from go to Google and paste in the following code there and you'll get a ton of results.
    Code:
    proxy ":8080" ":3128"
    You can do a number of things from there to filter/narrow your results down a bit, but that should give you nice list to start with of links you can grab some proxies from.
     
  5. bartosimpsonio

    bartosimpsonio Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 21, 2013
    Messages:
    8,911
    Likes Received:
    7,511
    Occupation:
    ZLinky2Buy SEO Services
    Location:
    ⇩⇩⇩⇩⇩⇩⇩⇩⇩⇩⇩⇩
    Home Page:
    About your anonymity question, it's pretty simple.

    Some proxies add X-ORIGINAL-IP(or something like that) header and then identify you as the source of the request. Thus you get people asking "I used a proxy and google said automated requests anyway". That's right, with that header on, the request came from your IP, not the proxy as far as they care.....
     
  6. SPPChristian

    SPPChristian Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 20, 2012
    Messages:
    1,221
    Likes Received:
    239
    Location:
    United States
    Home Page:
    you are right china proxies are not good for scrapping at all. If you are looking for list of public proxies there is a guy here called @proxygo contact him.
    Abut the socks, they can not be faster then http/https because the servers load is higher due to encryption, the speed really depends on peering from your ISP to the server network where the proxy is hosted, how far the server where the proxy is hosted is, and so on ...
    If you are looking for high speed private proxies that have a UP-Time of 99.99%, take a look at my signature
     
    Last edited: Oct 19, 2013
  7. proxygo

    proxygo Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 2, 2008
    Messages:
    10,295
    Likes Received:
    8,714
    if you use public proxies, you will have to replace them a couple of
    times per day

    if you use private / shared proxies provided you keep your requests down
    they should last a month, the more aggressive your searches the faster the
    proxie is banned