Verify domain names have a real website up and running

Hey!

Does anyone know of way to take a list of 30,000 domain names and verify that each domain has real, live, up and running website on it?

I guess it would like a mail verifier but instead of just checking for an mx record it would actually hit the domain and report back if a page loaded
or if there was an error like "page not found".

Using an email verification program won't help because it's easy enough to register a domain, setup email for it, and not have a website.

Any thoughts????

Thanks!

Scrapebox can do this I think. There are most likely other programs that could too, but I can't think of any at the moment.

Scrapebox has an "alive check" function where you set up certain HTTP response codes to report the URL as alive/dead.

It returns anything that does return an HTTP response of 301 or 200 as "dead" and the others as alive. You can customize this though.

That however is only going to be one step of the process, as plenty of your domains are going to be "parked" domains.

After performing the above function and getting a list of domains that return as "alive" (200 response code) you'd need to perform another function to determine a live website...this is tricky though. A parked domain will return a response code of 200 even though there is not actually a website there. There isn't a form of AI sufficient enough to "look" at a website and tell you "yes, this is a real website" or "no, this isn't a real website".

So you have to do something kind of "hacky" like either look for the word "parked" on the page or, another thought that ran through my head, check to see if there is a scrollbar present or not (thinking that if there is a scrollbar, it's a real website, if not, no website). Neither of those would return as 100% accurate though of course...just a few thoughts I had.

That's awesome - I thought for sure I was stuck with spending a month viewing all these websites!

Thank you both - looking Scrapebox over and it will definitely help - and has a ton of other uses too.

Thanks!

SB is a very useful tool, lots of functionality. There is a thread here where you can get it for half price. The link is at the bottom of the post.

Are there any free versions floating around out there?

Trident50 said:

Are there any free versions floating around out there?

Click to expand...

Unfortunately not, but it is well worth the price for all the features that it comes with.

You can get SB for free man, its one of the top Tools in BH SEO .