Any tool to scrape urls from google blog search engine?

Ruriko

Power Member
Joined
May 4, 2008
Messages
717
Reaction score
26
Anyone know a tool that can scrape urls from google's blog search engine?
 
What would be nice is to have a scrape tool to scrape URLS from the Do-Follow search engines.
 
'*******************************************************************
'Scrape a page's content into a string
Public Function GetHttp(Byval Url)
Dim objXml
Set objXml = createobject("MSXML2.ServerXMLHTTP")
objXml.open "GET", URL, false
objXml.send()
If Err <> 0 then
GetHttp = "Error: " & err.description
End If
if len(objXml.ResponseText) > 0 then
GetHttp = objXml.ResponseText
else
GetHttp = "There is no content."
end if
Set objXml = Nothing
End Function
'*******************************************************************
'Get the content to display based on a regExp pattern
Public Function GetContent(strContent, strPattern)
Dim objRegExp, Matches
Set objRegExp = New RegExp
objRegExp.Pattern = strPattern
objRegExp.IgnoreCase = True
objRegExp.Global = True
Set Matches = objRegExp.Execute(strContent)
for each match in Matches
GetContent = GetContent & match.Value
next
Set objRegExp = Nothing
End Function

'example:
dim strURL : strURL = "urltoscrapehere"
strFullContent = GetHttp(strURL)
response.write(GetContent(strFullContent,"<a href=(.*?)>"))






this is old and is in classic asp but i have used script like this many times to do exactly what your looking to do. I hope that this can help you out. You might need to spoof to make it look like it's coming from a browser for google. If you need any more help gettingthis going PM me and I can see what I can find.
 
scrape genius only uses google search engine. I want to use the google blog search engine
 
Back
Top