Help with Scrapebox Alive Check addon

[dcuthbert]

Hi,

I had a list of about 10k URLs (all subfolders within the same site, ie domain.com/folder1, domain.com/folder2 etc) and want to check which URLs exist and which don't. I ran the list through Scrapebox's Alive Check addon and it returned about 7k of those as being 'alive', and 3k 'dead'.

I didn't expect more than a few hundred to be dead so I checked some manually, and all of the ones I've checked which Scrapebox marked as dead (about 50) do exist and have content posted on them. Is there any other way I can check whether any of these are actually sitting empty? I don't fancy having to check 3k URLs manually!

 

[gullsinn]

Load your SB with a lot of active proxies and if you have a fast internet connection then make the number of thread more otherwise make the number of threads less and try again.

 

[bendutchman]

If your just checking if urls are alive it's best not to use proxies as they are not needed for this process.

 

[dcuthbert]

Load your SB with a lot of active proxies and if you have a fast internet connection then make the number of thread more otherwise make the number of threads less and try again.

If your just checking if urls are alive it's best not to use proxies as they are not needed for this process.

Thanks both. I found the problem - the site I was checking must have changed their URL structure at one stage, so the URLs were 301'd to a slightly different URL. Scrapebox must recognise 301'd URLs as dead which was why I was finding this.

 

[alaltaierii]

You should try Xenu link sleuth for checking dead sites.

 

[Sweetfunny]

Thanks both. I found the problem - the site I was checking must have changed their URL structure at one stage, so the URLs were 301'd to a slightly different URL. Scrapebox must recognise 301'd URLs as dead which was why I was finding this.

If you click the "Options" button you can set it to class 301, 302 redirects or any other response code as alive or dead. You can also enable "Follow Relocation" so if it encounters a 301 or 302 redirect it can even follow the redirects and tell you if the destination URL is alive or dead.

It can be completely customized how you want it.

 

[dodo34]

scrapebox shows my domain dead, but my website is online. why thats happening ?

 

[dodo34]

please help. Thanks