Scraping keywords from 50.000 domains

Discussion in 'General Scripting Chat' started by mrmystic, Sep 18, 2014.

  1. mrmystic

    mrmystic Newbie

    Joined:
    May 1, 2014
    Messages:
    24
    Likes Received:
    1
    Hello. I have a domainlist on 50.000 domains.

    I want to find out what category these websites are in, like are they doing sports, computer, webdesign and so on.
    Is there a free way i can 'scrape' or find out who uses the keyword or word sport on their website and so?

    Like type: Sport, and then it shows me all the domains which contains sports on their website
     
  2. TrafficWizard

    TrafficWizard Junior Member

    Joined:
    Aug 22, 2014
    Messages:
    164
    Likes Received:
    28
    Gender:
    Female
    I will write a script for you.. !

    However I have no effin clue how to get keyword/category idea out of sites.. could I just scrape all meta description ?
    <meta name="description" content="Health and Fitness products, health and fitness product reviews" />

    This could work or you need something else ? PM
     
  3. member8200

    member8200 Regular Member

    Joined:
    Aug 9, 2014
    Messages:
    475
    Likes Received:
    33
    I think the best option you can do is to hire a programmer to build the script for you, i suggest a php or python programmer.