1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

WHOIS scraper?

Discussion in 'Black Hat SEO' started by Zidane10, Nov 30, 2014.

  1. Zidane10

    Zidane10 Newbie

    Joined:
    Feb 19, 2014
    Messages:
    24
    Likes Received:
    6
    Hi. Does anyone know of a good whois scraper? The one on scrapebox isn't really good. I need one that will get all of the emails for a domain.
     
  2. CyberAlien

    CyberAlien Regular Member

    Joined:
    Apr 14, 2010
    Messages:
    483
    Likes Received:
    231
    You can whois all of the popular domains through command line. The fastest way would be to download a registry list of all the .com's and then use Perl to whois all of them and output the results to a file. You could then simply extract the emails from that file. If you're planning on scraping millions of domains, this would be the fastest way.
     
  3. Zidane10

    Zidane10 Newbie

    Joined:
    Feb 19, 2014
    Messages:
    24
    Likes Received:
    6
    I'm talking about 8000-10000 domains. I'm not familiar with perl. Is there a program that can do this?
     
  4. lord1027

    lord1027 Elite Member

    Joined:
    Sep 20, 2013
    Messages:
    3,174
    Likes Received:
    2,222
    You can do it in linux easily, with a small bash script. But I'm not sure exactly how to make it multithreaded.
     
  5. Mercury_Hg

    Mercury_Hg Registered Member

    Joined:
    Aug 23, 2010
    Messages:
    88
    Likes Received:
    18
    It doesn't sound like it'd be very difficult to accomplish in C++. Stream domains from file -> Query [multiple] whois domain -> retain emails -> stream results out to file

    It'd be pretty trivial to filter out masked emails too, i.e. those who have whois protection.
     
  6. ronalde

    ronalde Registered Member

    Joined:
    Aug 1, 2014
    Messages:
    79
    Likes Received:
    21
    Great custom footprints and then scrape with Gscraper.
     
  7. Zidane10

    Zidane10 Newbie

    Joined:
    Feb 19, 2014
    Messages:
    24
    Likes Received:
    6
    Can anyone create a program that will do this?
     
  8. Mercury_Hg

    Mercury_Hg Registered Member

    Joined:
    Aug 23, 2010
    Messages:
    88
    Likes Received:
    18
    I could. Do you have the domain names you need already?

    EDIT: This wouldn't need multithreading. If it's only 10,000 queries, you could accomplish them at a conservative speed of one query / sec in under three hours. That way you don't get IP banned for essentially DoSing the site.
     
    Last edited: Dec 1, 2014