1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How are 404 broken links detected with tools?

Discussion in 'Black Hat SEO' started by besupreme, Aug 9, 2014.

  1. besupreme

    besupreme Regular Member

    Joined:
    May 6, 2013
    Messages:
    253
    Likes Received:
    54
    Location:
    United Kingdom
    Hi guys,

    Just wanted ask if anybody knows how a 404 broken links get detected when running tools. What I'm trying to find out is how the tools find these links. I know they crawl them but I need to have a better answer than this. For example I've used seoProfiler and Screaming Frog to do a quick seo audit on the site and came across a broken link. the url is given but I'm not sure how it's found it as it's not a link on the navigation bar. I don't know if I've explained it well but to give you an example its found a broken link like:

    www.example.example1.com/en-GBR

    which shows as broken link, but this is could be due to that fact it should be

    www.example.example1.com/en-GBR/

    There is a canonical tag on the header which is <link rel="canonical" href="/en-GBR/">

    I just need to know how it's done this so that I can explain it in a report, rather than just say its a broken link but have have no answer to how the crawler has found this broken link.

    ​Thanks in advance :)



     
  2. divok

    divok Senior Member

    Joined:
    Jul 21, 2010
    Messages:
    1,015
    Likes Received:
    634
    Location:
    http://twitter.com/divok
    • Thanks Thanks x 1
  3. besupreme

    besupreme Regular Member

    Joined:
    May 6, 2013
    Messages:
    253
    Likes Received:
    54
    Location:
    United Kingdom
    Hey thanks for your help.

    So does that mean the broken link is from the server in which the GET method returned a 404 broken link?

    Also would it be safe to say that with this broken link its better to 301 redirect this url to resolve the broken link?
     
  4. Techxan

    Techxan Elite Member

    Joined:
    Dec 7, 2011
    Messages:
    3,093
    Likes Received:
    3,585
    Occupation:
    Local SEOist
    Location:
    TEXAS (you have to yell, its the law.)
    Down load Xenu link sleuth.

    It will tell you the origin and destination of the link.
     
    • Thanks Thanks x 1
    Last edited: Aug 10, 2014
  5. Conor

    Conor Jr. VIP Jr. VIP

    Joined:
    Nov 7, 2012
    Messages:
    3,358
    Likes Received:
    5,420
    Gender:
    Male
    Location:
    South Africa
    Home Page:
    Yes .
     
    • Thanks Thanks x 1
  6. besupreme

    besupreme Regular Member

    Joined:
    May 6, 2013
    Messages:
    253
    Likes Received:
    54
    Location:
    United Kingdom
    Thanks for the recommendation but I've already used "Xenu" :) . I just need to know how to the tool managed to find the broken link, so that I can write it up in a report. ( I can't just list that my tool found a broken link but not explain the process on how it managed to find it).

    Thanks for you help again I do appreciate it.