1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Find Expired Domains in the DNS Census from 2013

Discussion in 'Black Hat SEO' started by regicider, Feb 5, 2015.

  1. regicider

    regicider Junior Member

    Joined:
    Apr 2, 2012
    Messages:
    109
    Likes Received:
    54
    For those of you building PBN's, here's something I came across. In 2013, there was an initiative to perform a census of all the domains in existence. Although they didn't get everything, they amassed a huge collection of domains - over 150 million - all organized by TLD. I've processed a sample of these, and there are definitely some worthwhile expired domains in the bunch. However, since I'm not actively building a PBN, and I'm not a domain reseller, I thought I'd pass along the info.

    Heres a link to the torrent for the whole dataset: (it's huge, around 12 GB I think, and if you're on Windows, you'll need 7zip to unpack the compressed files)

    Code:
    http://1drv.ms/16I1OFL
    
    Code:
    Virus Total:
    https://www.virustotal.com/en/url/9003550aa1256bc5c87b141e83f8bb35f549f8ccd0f1ab718d72a355ef42dd0b/analysis/1423165418/
    
    Here's a tip if you start processing these - skip the domains at the beginning of the file that start with 0-9. It's almost universally garbage, you'll get better results with domains that start with letters.
     
    • Thanks Thanks x 1
  2. tutughosh

    tutughosh Jr. VIP Jr. VIP

    Joined:
    Sep 28, 2010
    Messages:
    1,591
    Likes Received:
    485
    Occupation:
    Business
    Location:
    Some Where in the Optical
    Home Page:
    This is really a Good Share Buddy.. 14.5GB of DATA.. Start Downloading already.. :)
     
  3. karupoiss

    karupoiss Elite Member

    Joined:
    May 16, 2012
    Messages:
    1,866
    Likes Received:
    593
    Gender:
    Male
    Location:
    Europe
    Thanks for the list but don't you think people have already scanned it through multiple times if the list was created in 2013?
     
  4. HelloInsomnia

    HelloInsomnia Jr. Executive VIP Jr. VIP

    Joined:
    Mar 1, 2009
    Messages:
    1,826
    Likes Received:
    2,939
    Possibly, but it's not a bad idea overall. You can get zone file access from Verisign for the .com and .net zone files then just download them once per month and look at the previous months (will have to wait the first month) for a fresher list.

    I won't say exactly how to do this but if you look for yourself you will find it's not actually that difficult.
     
    • Thanks Thanks x 1
  5. regicider

    regicider Junior Member

    Joined:
    Apr 2, 2012
    Messages:
    109
    Likes Received:
    54
    People have absolutely scanned through it multiple times. But, since domains get dropped all the time, someone who scanned it last month will get different results from doing it now. Obviously it's not a small task combing through these, and there will be a large amount useless domains (this is true for any method of acquiring expired domains). But due to the sheer volume of domains here, there are still many, many valuable domains to be found.
     
  6. rogerke

    rogerke Regular Member

    Joined:
    Oct 5, 2014
    Messages:
    264
    Likes Received:
    145
    Thanks for the share. Will look into it when the (slow) torrent is finally finished. I suppose it only includes domains which were (still) registered in 2013, right? It'd be awesome if it includes all domains ever registered, but I don't expect that.

    P.S. might want to delete that map with the expired domain scraper . ;)
     
    Last edited: Feb 6, 2015
  7. regicider

    regicider Junior Member

    Joined:
    Apr 2, 2012
    Messages:
    109
    Likes Received:
    54
    You're welcome. I'm not sure how the domains were collected, but I'm pretty certain it's not all domains ever -however it could contain domains that were dropped before the data was collected.