How to check multiple domains for 404 pages (in bulk)
- Thread starter fedez
- Start date
HelloBTCMINER
BANNED
- Jun 21, 2018
- 356
- 388
Try this - https://httpstatus.io/
This is good but it only checks the main URL. If I put in xyz.com I want to check every page
xyz.com/page1
xyz.com/page2
xyz.com/page3
xyz.com/page4
for 404 errors
D
Deleted member 923340
Guest
Are you familiar with any programming language? You can do this with anything. For example using java.
Code:
public static boolean doesURLExist(URL url) throws IOException
{
// We want to check the current URL
HttpURLConnection.setFollowRedirects(false);
HttpURLConnection httpURLConnection = (HttpURLConnection) url.openConnection();
// We don't need to get data
httpURLConnection.setRequestMethod("HEAD");
// Some websites don't like programmatic access so pretend to be a browser
httpURLConnection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)");
int responseCode = httpURLConnection.getResponseCode();
// We only accept response code 200
return responseCode == HttpURLConnection.HTTP_OK;
}