1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Looking for a domain crawler

Discussion in 'General Programming Chat' started by jimmyt12345, Dec 18, 2014.

  1. jimmyt12345

    jimmyt12345 Newbie

    Joined:
    Dec 18, 2014
    Messages:
    3
    Likes Received:
    0
    Hi anyone know of a crawler that can go through a list of uploaded domains to see if they are parked/active

    Help with this is much appreciated!!

    Thanks
     
  2. FBGuru

    FBGuru Senior Member

    Joined:
    Sep 22, 2013
    Messages:
    928
    Likes Received:
    1,172
    Location:
    Personality Type : ESTP
    If it's for checking expired domains, you can try Scrapebox's alive-checker addon.
     
  3. Aty

    Aty Jr. VIP Jr. VIP

    Joined:
    Jan 27, 2011
    Messages:
    5,990
    Likes Received:
    4,083
    Occupation:
    SEO (Senior Erection Officer)
    Location:
    your 6 o'clock
    Home Page:
  4. JournoNick

    JournoNick Power Member

    Joined:
    Feb 11, 2014
    Messages:
    613
    Likes Received:
    311
    Location:
    on the frontline
    Care to give some more detail on what you want? Why would you want to check if domains are alive?
     
  5. jimmyt12345

    jimmyt12345 Newbie

    Joined:
    Dec 18, 2014
    Messages:
    3
    Likes Received:
    0
    Basically i have an old directory database that i want to check before i put it live again on my site however i was going through a lot of the domains and they now seem to be parked hence i need a tool that can recognize if the site is live or now parked or dead.

    i tried using xenu but I'm not quite sure it will download the results in mass seems perfect for small lists?
     
  6. RuthSam

    RuthSam Jr. VIP Jr. VIP Premium Member

    Joined:
    Mar 19, 2010
    Messages:
    3,813
    Likes Received:
    976
    Gender:
    Male
    Home Page:
    Uou may use the search field on this forum "domain crawler" there is a lady who is providing such software which works great and hassle free for this purpose.

    I got some outstanding and great support from Scrapebox to do something similar, but I can't get it work and it also sounds like it can't do the same 100% at all.
     
  7. mindmaster

    mindmaster Jr. VIP Jr. VIP

    Joined:
    Sep 16, 2010
    Messages:
    2,765
    Likes Received:
    1,240
    Home Page:
    • Thanks Thanks x 1
  8. prab1996

    prab1996 Elite Member

    Joined:
    Jan 8, 2013
    Messages:
    3,493
    Likes Received:
    2,031
    Occupation:
    your gf's <3 ♥♥♥♥
    Location:
    Prab1996.com
    Home Page:
    I will do that for you for free. just send me list. you will get list of alive domains back.
    -=-
     
  9. overy

    overy BANNED BANNED

    Joined:
    Oct 30, 2015
    Messages:
    41
    Likes Received:
    2
    You can check the domain registration, expiration and other whois information by using this free tool