1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[need] Suggestions for a site crawler

Discussion in 'Black Hat SEO' started by schmidcpc, Oct 30, 2010.

  1. schmidcpc

    schmidcpc Newbie

    Joined:
    Feb 15, 2010
    Messages:
    6
    Likes Received:
    0
    I am looking for a crawler that will return ALL URL's for a specified site. The site I need crawled has about 30k urls.

    Thank you for your suggestions, (free utility preferably)