1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Fastest way to harvest relevant URLs

Discussion in 'Black Hat SEO Tools' started by postcd, Feb 23, 2010.

  1. postcd

    postcd BANNED BANNED

    Joined:
    Nov 14, 2009
    Messages:
    145
    Likes Received:
    7
    Hi,

    Im using URL Harvester 2008 to harvest URLs from the net.

    It saves URLs from Google, but the problem is that google ban you when accessing it too many times per time period.

    So my question is what software or method You are using to harvest URLs from the net. What is the fastest method?

    Thank you much
     
  2. jesdon2

    jesdon2 Junior Member

    Joined:
    Jun 19, 2009
    Messages:
    122
    Likes Received:
    85
    Buy Scrapebox. You can scrape thousands of urls, with custom footprints, using random proxies and then remove all duplicates and sort the list by pagerank. Very useful tool.
     
    • Thanks Thanks x 1
  3. liangtvdotcom

    liangtvdotcom Newbie

    Joined:
    Dec 27, 2009
    Messages:
    49
    Likes Received:
    0
    For example I wana scrape all the URLs links from www. xxx . com what keyword should i type?

    "www. xxx . com /urls/"
    or
    site:www . xxx .com / urls ?

    using scapebox? i tried alot of ways but cant get thousands of them
     
  4. keverone

    keverone Junior Member

    Joined:
    Oct 4, 2009
    Messages:
    147
    Likes Received:
    45

    You can ... just go get scrapbox it is sold on BHW search it out..
     
  5. ericsson

    ericsson Elite Member Premium Member

    Joined:
    Apr 25, 2009
    Messages:
    2,642
    Likes Received:
    8,132
    Occupation:
    www
    Location:
    Swe
    Home Page:
    Ok, here´s the deal.

    Hehe! Don´t use to many threads at the same time and you will be good.
    Im using 10-15 with url harvester 2008. And google will not ban you, you just need to wait 3-4 hours before harvest again. Stick to 10-15 multi threads, and you will be fine.

    //J
     
    • Thanks Thanks x 1