1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Crawl bot with no limits

Discussion in 'Black Hat SEO Tools' started by CrAzYPiLoT, Apr 27, 2016.

  1. CrAzYPiLoT

    CrAzYPiLoT Newbie

    Apr 27, 2016
    Likes Received:
    Hey, so I need a crawler bot which can list all of the pages on a site, even if they don't have any links to them. For example, an user gets an email with a link to some page, and only he has the link, possibly the admin too, but there are no links to the page on other pages. Does something like this exist? Thanks in advance. Also, yes, the page I need to get to is disallowed in robots.txt and is probably not indexed.