1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Help with Scrapebox Alive Check addon

Discussion in 'Black Hat SEO Tools' started by dcuthbert, Jul 21, 2013.

  1. dcuthbert

    dcuthbert Regular Member

    Joined:
    Jun 15, 2011
    Messages:
    411
    Likes Received:
    249
    Hi,

    I had a list of about 10k URLs (all subfolders within the same site, ie domain.com/folder1, domain.com/folder2 etc) and want to check which URLs exist and which don't. I ran the list through Scrapebox's Alive Check addon and it returned about 7k of those as being 'alive', and 3k 'dead'.

    I didn't expect more than a few hundred to be dead so I checked some manually, and all of the ones I've checked which Scrapebox marked as dead (about 50) do exist and have content posted on them. Is there any other way I can check whether any of these are actually sitting empty? I don't fancy having to check 3k URLs manually!
     
  2. gullsinn

    gullsinn Jr. VIP Jr. VIP Premium Member

    Joined:
    Dec 24, 2009
    Messages:
    2,429
    Likes Received:
    2,210
    Gender:
    Male
    Occupation:
    Jobless :D
    Location:
    Graveyard
    Home Page:
    Load your SB with a lot of active proxies and if you have a fast internet connection then make the number of thread more otherwise make the number of threads less and try again.
     
    • Thanks Thanks x 1
  3. bendutchman

    bendutchman Junior Member

    Joined:
    Jun 1, 2012
    Messages:
    131
    Likes Received:
    41
    Occupation:
    genetic engineer
    Location:
    House, Road House
    If your just checking if urls are alive it's best not to use proxies as they are not needed for this process.
     
    • Thanks Thanks x 1
  4. dcuthbert

    dcuthbert Regular Member

    Joined:
    Jun 15, 2011
    Messages:
    411
    Likes Received:
    249
    Thanks both. I found the problem - the site I was checking must have changed their URL structure at one stage, so the URLs were 301'd to a slightly different URL. Scrapebox must recognise 301'd URLs as dead which was why I was finding this.
     
  5. alaltaierii

    alaltaierii Supreme Member

    Joined:
    Jun 11, 2010
    Messages:
    1,408
    Likes Received:
    349
    You should try Xenu link sleuth for checking dead sites.;)
     
  6. Sweetfunny

    Sweetfunny Jr. VIP Jr. VIP Premium Member

    Joined:
    Jul 13, 2008
    Messages:
    1,747
    Likes Received:
    5,039
    Location:
    ScrapeBox v2.0
    Home Page:
    If you click the "Options" button you can set it to class 301, 302 redirects or any other response code as alive or dead. You can also enable "Follow Relocation" so if it encounters a 301 or 302 redirect it can even follow the redirects and tell you if the destination URL is alive or dead.

    It can be completely customized how you want it.
     
    • Thanks Thanks x 2
  7. dodo34

    dodo34 Newbie

    Joined:
    Dec 16, 2013
    Messages:
    37
    Likes Received:
    3
    scrapebox shows my domain dead, but my website is online. why thats happening ?
     
  8. dodo34

    dodo34 Newbie

    Joined:
    Dec 16, 2013
    Messages:
    37
    Likes Received:
    3
    please help. Thanks