1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Crawl list of urls, collect all internal links - how?

Discussion in 'Black Hat SEO Tools' started by Satyren, Apr 10, 2015.

  1. Satyren

    Satyren Regular Member

    Joined:
    Nov 29, 2009
    Messages:
    496
    Likes Received:
    598
    Location:
    EU
    Yo.

    I have a list of urls I'd like to crawl to collect all its internal links. Do you guys know a tool that can do this? I don't have the sitemaps, so is guess scrapebox probably isn't an option.

    Thanks in advance,
    s.
     
  2. rogerke

    rogerke Regular Member

    Joined:
    Oct 5, 2014
    Messages:
    264
    Likes Received:
    145
    Xenu, Screaming Frog and Scrapebox are all capable of doing that.
     
  3. Satyren

    Satyren Regular Member

    Joined:
    Nov 29, 2009
    Messages:
    496
    Likes Received:
    598
    Location:
    EU
    Just downloaded Xenu looks great. Thank you man :)
     
  4. yladiator

    yladiator Newbie

    Joined:
    Apr 2, 2015
    Messages:
    7
    Likes Received:
    0
    it's very important inf
     
  5. xrfanatic

    xrfanatic Jr. VIP Jr. VIP

    Joined:
    Aug 28, 2010
    Messages:
    404
    Likes Received:
    173
    Location:
    http://bit.ly/slb64
    Home Page:
    Seems like you have to take a look at my sig ;-)

    We are currently giving away 10 freebie licenses of SEO List Builder so obtaining a copy will cost you responding in thread below :)

    http://www.blackhatworld.com/blackh...-licenses-freebie-giveaway-4.html#post7815500

    Just post in the thread and I will PM you with the 100% free license key so you can scrape the internal links of your pages

    If you have any questions I can help you with setting it properly :)