1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Proxies for web scraping

Discussion in 'Proxies' started by blackscrapy, May 15, 2015.

  1. blackscrapy

    blackscrapy Newbie

    Joined:
    Oct 30, 2014
    Messages:
    36
    Likes Received:
    2
    Hello

    I am in the lookout for some reliable shared proxies that would be used for web scraping.

    Kindly suggest from where I can get these proxies


    Thanks
     
  2. blackscrapy

    blackscrapy Newbie

    Joined:
    Oct 30, 2014
    Messages:
    36
    Likes Received:
    2
    Anyone?

    Also please advise whether shared or dedicated (private) would be better for web scraping?
     
  3. SpamBotKing

    SpamBotKing Registered Member

    Joined:
    May 15, 2015
    Messages:
    62
    Likes Received:
    128
    Hei, SpamBotKing here.
    Sadly, I cannot give to you any info about where buy proxy (I use a private seller right now) but I think any proxy vendors will be fine.

    * If I have time, and the scraping process is NOT confidential (example: scan a craiglist like website etc.) I go with public proxy. I scrape them, check them, keep the most quicks, and then go. Not an elegant way, but it works.
    * If I need to scrape a HUGE amounts of page of a website and I have time, I go with shared
    * If I have to maximize my operations, I will use private proxies.
     
    • Thanks Thanks x 1
  4. seeplusplus

    seeplusplus Power Member

    Joined:
    Aug 18, 2008
    Messages:
    517
    Likes Received:
    164
    Depends on what sites your scraping. Google best with private, some random site you might not need any.
     
    • Thanks Thanks x 1
  5. Paper-Boy

    Paper-Boy Elite Member

    Joined:
    Jun 17, 2009
    Messages:
    5,118
    Likes Received:
    1,823
    For scraping I would use proxies that come with ports. It changes proxies every 10 minutes.

    So technically you can use 100s of proxies without the fear of getting them banned on google.
     
    • Thanks Thanks x 1
  6. blackscrapy

    blackscrapy Newbie

    Joined:
    Oct 30, 2014
    Messages:
    36
    Likes Received:
    2
    Thanks for your replies guys. I would be using these proxies mainly for scraping YellowPages (US/AU) I tried some shared proxies but they were too slow. I signed up for Reverse Port proxies, but the vendor never responded, so I canceled.

    I am looking for a provider who provides quality proxies (anonymous) at reasonable prices and offer great support.

    Any suggestions?
     
  7. ShaNi

    ShaNi Jr. VIP Jr. VIP

    Joined:
    Aug 12, 2013
    Messages:
    1,243
    Likes Received:
    407
    Location:
    Red-Proxy. LLC
    Home Page:
    We're offering our US Private proxies for web Scraping :)
    but which web Scraping sites you want ? and which tool
    you using ?you can check on our signature with dedicated
    Skype - email Customers Support
     
  8. blackscrapy

    blackscrapy Newbie

    Joined:
    Oct 30, 2014
    Messages:
    36
    Likes Received:
    2
    Hi Shani

    Thanks for your message

    As I said, I primarily am looking to scrape YellowPages (US & AU) with it and some other sites as and when needed.

    The scraping tools are custom made in house with C#

    Can I get a review copy to test out your proxies first?


    Thanks
     
  9. ShaNi

    ShaNi Jr. VIP Jr. VIP

    Joined:
    Aug 12, 2013
    Messages:
    1,243
    Likes Received:
    407
    Location:
    Red-Proxy. LLC
    Home Page:
    then its work but where you using your tool on normal DSL Speed
    or VPS = Dedicated Server ? sorry no trial proxies :) Private proxies
    must be work
     
  10. blackscrapy

    blackscrapy Newbie

    Joined:
    Oct 30, 2014
    Messages:
    36
    Likes Received:
    2
    I run them mostly from my desktop with a 1Mbps cable internet connection.
     
  11. ShaNi

    ShaNi Jr. VIP Jr. VIP

    Joined:
    Aug 12, 2013
    Messages:
    1,243
    Likes Received:
    407
    Location:
    Red-Proxy. LLC
    Home Page:
    You using low DSL Speed for Scraping mate . your fist method is wrong
    Scraping means Power . You need nice VPS or Dedicated Server machine
    with 1GB port link Speed . if you using low DSL Speed then you can't see
    nice results on any types of Proxies

    always use all SEO customs tools or famous Scraping tools on nice Vps
    or Dedicated Server machine's then you will See awesome results ;)
    if you Scraping normal just few 100s results then your 1MB is just okay
    but still not good
     
  12. blackscrapy

    blackscrapy Newbie

    Joined:
    Oct 30, 2014
    Messages:
    36
    Likes Received:
    2
    Thanks for the suggestion, Shani :)
     
  13. extremeboy

    extremeboy Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2010
    Messages:
    3,185
    Likes Received:
    667
    Occupation:
    World Best RANK Tracker SERPCloud.com
    Home Page:
    @blackscrapy:

    I much experienced on web scraping from search engines if you still want some tips on proxies where to get and how to use and what are major steps to take to get done.

    Let me know :)

    Thanks!
     
  14. saurav560

    saurav560 Newbie

    Joined:
    May 19, 2015
    Messages:
    9
    Likes Received:
    2
    I have used buyproxies for the same and they do a good job.
     
  15. blackscrapy

    blackscrapy Newbie

    Joined:
    Oct 30, 2014
    Messages:
    36
    Likes Received:
    2

    That would be great extremeboy

    Please suggest on the best possible tools, proxies and practices for web scraping.

    I don't scrape too much of Google, but mainly sites like YellowPages, YellowBot etc

    Your valuable advice is solicited :)


    Thanks !
     
  16. KC1987

    KC1987 Registered Member

    Joined:
    Mar 6, 2015
    Messages:
    50
    Likes Received:
    13
    I use buyproxies org for my gsa ser, scrapebox and kontent machine activities
     
  17. extremeboy

    extremeboy Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2010
    Messages:
    3,185
    Likes Received:
    667
    Occupation:
    World Best RANK Tracker SERPCloud.com
    Home Page:

    Add me on skype then i can give my advice for you according to your resources you have thanks :)
     
  18. DrFreemna

    DrFreemna Registered Member

    Joined:
    May 1, 2015
    Messages:
    52
    Likes Received:
    8
    Occupation:
    Software developer
    Location:
    Serbia
    HMA updates their public proxy list every minute with 7-10 new entries, that should be more that enough. I've used them quite a bit and I haven't run on any issues.
    If you have trouble scraping proxy list from HMA website, let me know, I could probably help you.

    Good luck.
     
  19. blackscrapy

    blackscrapy Newbie

    Joined:
    Oct 30, 2014
    Messages:
    36
    Likes Received:
    2
    The proxies I am currently using are good but they don't allow connections via ports 2082, 2083 and 25

    Does anyone know of a proxy dealer who doesn't block the above ports?