1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to scrape tons of sites with few footprints?

Discussion in 'Black Hat SEO' started by directaxcess, Oct 4, 2012.

  1. directaxcess

    directaxcess Power Member

    Joined:
    Sep 12, 2011
    Messages:
    780
    Likes Received:
    154
    Say you're trying to scrape for PHPld directories. Now, there's only a certain # of footprints that are common amongst all directories ... however Google will only allow you to access the first 1000.

    If that's the case, how can we scrape thousands and thousands of web directories (or any platform for that matter) with only a few footprints? Looking forward to your responses.