Is this Possible?

Discussion in 'General Programming Chat' started by mishiefmade, Sep 20, 2010.

Tags:
  1. mishiefmade

    mishiefmade Newbie

    Joined:
    Jul 7, 2009
    Messages:
    13
    Likes Received:
    18
    Does anybody know if it is possible to create a code/script that logs into a website, scrapes a dynamic page and generates an rss feed from that pages content?

    The idea is to create an rss feed from a blog/listings page that automatically updates whenever new content is posted.

    i would really appreciate any advice/directions of how i could do this/if it is possible.

    Thanks
     
  2. smack

    smack Junior Member

    Joined:
    Feb 1, 2010
    Messages:
    182
    Likes Received:
    80
    Occupation:
    Software Engineer/Evil Genius
    Location:
    inside .NET
    yes, that is absolutely possible and could be accomplished fairly easily with most modern languages.
     
  3. marquist

    marquist Newbie

    Joined:
    May 13, 2010
    Messages:
    15
    Likes Received:
    3
    Google Reader does something similar to this when you subscribe to a site that doesn't have an RSS feed, although obviously it's generic. The general approach would be to periodically fetch the target page, parse it, regenerate the RSS feed for the [n] most recent items. It's not too hard, RSS is just a simple XML format. As for extracting the information, there are libraries like "scrapy" for Python, or you can roll your own.