1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What is the Best Tool and Best Dedicated Proxies for Scraping URLs from Google?

Discussion in 'Black Hat SEO' started by Tomwinners1, Aug 8, 2014.

  1. Tomwinners1

    Tomwinners1 Registered Member

    Joined:
    Feb 8, 2012
    Messages:
    66
    Likes Received:
    2
    Hello,

    Hello everyone,

    I have been using scrapebox to scrape URLs from Google for about 2 years. Without proxies, I always got 1000 results if I set the "results per keyword" to 1000. But now I have some bigger projects, so I need to use proxies to scrape URLs.

    I have tried some private proxies (usually 30 or 50) from different providers, like NewIPNow and Buyproxies(org), but the result is not so good, I can only get about 6k URLs for 10 keywords (1k per keyword so it should be 10k). By the way, I set the connection to 3, and the timeout time is 30 seconds.

    Currently, the scrapebox is running on a VPS, the Internet connection is quite good. And I need to scrape hundreds of campaigns now, about 20 keyword each campaign, and I am using the automation Add-on to scrape URLs automatically, there is a half-an-hour cooling time between each campaign.

    I want to get as many URLs per keyword as possible.

    So are there any other URL scraping tool is better than Scrapebox?

    And what is the best private proxy provider you have used?

    Looking forward to your reply! Thanks in advance.

    (PS: If there is anyone who can provide professional URL-scraping service, I would be interested, just PM me)
     
  2. ija61

    ija61 Senior Member

    Joined:
    Mar 2, 2011
    Messages:
    960
    Likes Received:
    634
    Gender:
    Male
    Occupation:
    The first SEO economist:)
    Location:
    Romania
    Home Page:
    Maybe I'm missing something here... how you can scrape without proxy with 3 connections and not get banned?

    Do you take a look at those 10 kw to see how many result are showed in G?

    Or you could load the 10 kw and use single thread harvester to see what you get.

    I'm sure that the problem is not coming from the proxy but from your kw's. Second you should use a 1:10 or 1:15 ratio between connection and proxy.
     
  3. lover

    lover Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 25, 2007
    Messages:
    2,147
    Likes Received:
    811
    Location:
    BHW
    Go with Gscraper or Hrefer, they are lot faster than Scrapebox.

    I personally use Gscraper with lot of Public proxies and get lot of scrapped urls per day.
     
  4. thachcaodotcom

    thachcaodotcom Junior Member

    Joined:
    Oct 21, 2009
    Messages:
    121
    Likes Received:
    151
    I use $5 DO vps with multi Torr, rotate IPs every 5 min. Work perfectly with SB
     
  5. DannyZhang

    DannyZhang Regular Member

    Joined:
    Apr 2, 2014
    Messages:
    233
    Likes Received:
    69
    Scrapebox can scrape peanuts compared to Gscraper or Hrefer. I suggest you have a look at those two if you want to get serious about scraping.
     
  6. SerpEvil

    SerpEvil Supreme Member

    Joined:
    Jan 1, 2012
    Messages:
    1,494
    Likes Received:
    445
    Gender:
    Male
    Occupation:
    WebMaster
    Location:
    Top 3 On Google
    gscraper is the best tool for scraping
     
  7. TheLinksLab

    TheLinksLab Registered Member

    Joined:
    Jun 27, 2014
    Messages:
    79
    Likes Received:
    35
    Occupation:
    Various Web Projects
    Location:
    Swansea, Wales, UK
    I use Gscraper personally, it does it's job well in my opinion.
     
  8. ggguuuttt

    ggguuuttt Junior Member

    Joined:
    Aug 2, 2014
    Messages:
    104
    Likes Received:
    12
    Check out buyproxy