What Data Would You Scrape?

Discussion in 'BlackHat Lounge' started by simplegnome, Aug 10, 2012.

Tags:
  1. simplegnome

    simplegnome Newbie

    Joined:
    May 1, 2012
    Messages:
    0
    Likes Received:
    0
    Hey you guys,

    I am starting to think about writing a web app that scrapes data and while I have a few ideas, I really want to create something that will be of use to many rather than just a few.

    In the past I have mainly build private price checkers, twitter intelligence tools and keyword scrapers but I want to push myself to build something that everyone can use (fo free).

    The current ideas are -

    * Clickbank product analytics tool - Pulls the daily feed from CB and analyses rising products based on gravity change and ranking change
    * Amazon product tool - Similar to the CB idea. Picks rising product trends and carries out the keyword research to spot easy wins.

    As this is something I want to build to release for free, I was wondering if anyone had any ideas for scraper and more importantly a source(s) to scrape?
     
  2. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    112
    there are already a ton of keyword scrapers on the market. i personally want to see an alternative to kontentmachine.com -- i was going to think about writing something like this myself, but I'm a web developer not a desktop developer.
     
  3. simplegnome

    simplegnome Newbie

    Joined:
    May 1, 2012
    Messages:
    0
    Likes Received:
    0
    I like the thinking. It would be interesting to look at linguistic patterns used on forums/blogs etc and find genuine spintax ideas based on the findings.
     
  4. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    112
    i don't really know what they do, if they just integrate w/ a content spinner or not. That's all I would want. Something quick and easy to make my seo task creation a bit easier. Linguistics patterns in forums? That doesn't sound like something quick and easy.
     
  5. simplegnome

    simplegnome Newbie

    Joined:
    May 1, 2012
    Messages:
    0
    Likes Received:
    0
    Yeah maybe I am over thinking this $hit - the patterns would evolve over time but would depend on having a massive datastore of content to draw upon.