1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Anyone know of crawler software that extracts web addresses?

Discussion in 'White Hat SEO' started by webhell, Mar 3, 2014.

  1. webhell

    webhell Registered Member

    Joined:
    Apr 25, 2013
    Messages:
    78
    Likes Received:
    20
    I'm looking for something that will crawl a given website and return all the outbound URls.

    Either anything that is within ahref tags or any text that starts www.

    Thanks
     
  2. luc1fer

    luc1fer Junior Member

    Joined:
    Oct 17, 2013
    Messages:
    141
    Likes Received:
    52
    email spider maybe its old but free, use google you will find alot and if you not pay find torrents
     
  3. seocrab

    seocrab Senior Member

    Joined:
    May 2, 2013
    Messages:
    969
    Likes Received:
    825
    Occupation:
    seo freelancer
    Location:
    UK
    Screaming Frog SEO spider will do that for you. It's free for up to 500 pages per URL.