RSS Parsing and SQL

For those who are interested. I wont go into much of the details for the usefullness of this code but some of you my like it. I recently was asked to help with RSS parsing. The idea was to catolog RSS Feeds into a sql server. The following might come in handy for some of you. These are the two fucntions that I wrote to break down RSS Feeds.

Public Sub GetFeed()
　
' Bring back the Feed
Dim doc As New XPathDocument(rssURL)
　
Dim nav As XPathNavigator = doc.CreateNavigator()
Dim iter As XPathNodeIterator = nav.Select(rssSelect)
' Loop through the nodes
While iter.MoveNext()
' Get the data we need from the node
ProcessNode(iter.Current)
End While
　
End Sub 'GetFeed
Sub ProcessNode(ByVal lstNav As XPathNavigator)
Dim title As String = ""
Dim link As String = ""
Dim description As String = ""
Dim pubdate As String = ""
' Get the child nodes
Dim iterNews As XPathNodeIterator = lstNav.SelectDescendants(XPathNodeType.Element, False)
' Loop through the child nodes
While iterNews.MoveNext()
' Save the current Name
Dim rssName As String = iterNews.Current.Name
' Is this the title?
If rssName.ToUpper() = rssTitle.ToUpper() Then
title = iterNews.Current.Value
End If
' Is this the Link?
If rssName.ToUpper() = rssLink.ToUpper() Then
link = iterNews.Current.Value
End If
' Is this the Description?
If rssName.ToUpper() = rssDescription.ToUpper() Then
description = iterNews.Current.Value
End If
If rssName.ToUpper() = rssPubDate.ToUpper() Then
pubdate = iterNews.Current.Value
End If
End While
' Make sure the link and title are at least 10 characters long
If link.Length > 10 And title.Length > 10 Then
' Update the database
InsertNews(SourceID, link, title, description, pubdate)
End If
End Sub 'ProcessNode

For these functions to work you will need to populate a sql server table with the following fields:

SourceID
RSSSelect
RSSTitle
RSSLink
RSSDescrp
RSSURL
RSSpubDate

You can call the function simply like
GetFeed()

In my case the following is a simple example


Dim Dr2 As DataRow
For Each Dr2 In dt2.Rows
sql = "Delete TempNews from tempnews"
objCommand = New SqlCommand(sql, HomeServer)
objCommand.ExecuteNonQuery()
For Each dr In dt.Rows
SourceID = dr("SourceID").ToString()
rssSelect = dr("RSSSelect").ToString()
rssTitle = dr("RSSTitle").ToString()
rssLink = dr("RSSLink").ToString()
rssDescription = dr("RSSDescrp").ToString()
rssURL = dr("RSSURL").ToString() & Dr2("Symbol").ToString() & dr("RSSURLEND").ToString()
rssPubDate = dr("RSSpubDate").ToString()
symbol = Dr2("Symbol").ToString()
Console.WriteLine(rssURL)
GetFeed()
Next

The idea is to populate the sql server table with rss feed info. Populate the table with the tags that appear before each of the nodes you're interested in. Then you can break down that RSS feed into the info you need and bring it back in to a sql server for analysis. I wrote this up pretty quick so I'm sorry if my brian is scattered. If you need any help using the function or want to see more of the code let me know.