How to scrape tons of sites with few footprints?

Discussion in 'Black Hat SEO' started by directaxcess, Oct 4, 2012.

  1. directaxcess

    directaxcess Senior Member

    Sep 12, 2011
    Likes Received:
    Say you're trying to scrape for PHPld directories. Now, there's only a certain # of footprints that are common amongst all directories ... however Google will only allow you to access the first 1000.

    If that's the case, how can we scrape thousands and thousands of web directories (or any platform for that matter) with only a few footprints? Looking forward to your responses.