I need a tool that I can put in a domain (or better yet a list of domains) and it goes through all of them and finds all the 404 pages on each domain. So for example. I put in xyz.com
and it finds 2 404 pages on the website like
xyz.com/page1404
xyz.com/page2404
So far my only idea has been to find the sitemaps on domains and extract urls of whole site then check those for 404 or not, but this isnt good as not all sitemaps are easy to locate and many of them update regularly and dont include 404 pages.
I basically want to find all 404 pages on a domain that other domains are linking to. And if I can check a list of domains for all the 404 pages on each domain, even better. Can anyone program something like that for me?
Cheers
and it finds 2 404 pages on the website like
xyz.com/page1404
xyz.com/page2404
So far my only idea has been to find the sitemaps on domains and extract urls of whole site then check those for 404 or not, but this isnt good as not all sitemaps are easy to locate and many of them update regularly and dont include 404 pages.
I basically want to find all 404 pages on a domain that other domains are linking to. And if I can check a list of domains for all the 404 pages on each domain, even better. Can anyone program something like that for me?
Cheers
