1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Is this Possible?

Discussion in 'General Programming Chat' started by mishiefmade, Sep 20, 2010.

Tags:
  1. mishiefmade

    mishiefmade Newbie

    Joined:
    Jul 7, 2009
    Messages:
    13
    Likes Received:
    18
    Does anybody know if it is possible to create a code/script that logs into a website, scrapes a dynamic page and generates an rss feed from that pages content?

    The idea is to create an rss feed from a blog/listings page that automatically updates whenever new content is posted.

    i would really appreciate any advice/directions of how i could do this/if it is possible.

    Thanks
     
  2. smack

    smack Junior Member

    Joined:
    Feb 1, 2010
    Messages:
    182
    Likes Received:
    78
    Occupation:
    Software Engineer/Evil Genius
    Location:
    inside .NET
    yes, that is absolutely possible and could be accomplished fairly easily with most modern languages.
     
  3. marquist

    marquist Newbie

    Joined:
    May 13, 2010
    Messages:
    15
    Likes Received:
    3
    Google Reader does something similar to this when you subscribe to a site that doesn't have an RSS feed, although obviously it's generic. The general approach would be to periodically fetch the target page, parse it, regenerate the RSS feed for the [n] most recent items. It's not too hard, RSS is just a simple XML format. As for extracting the information, there are libraries like "scrapy" for Python, or you can roll your own.