Best scrapping tools

C

capflam

Junior Member

Aug 7, 2008

142

3

• Sep 1, 2009

• #1

I am looking for scrapping specific pieces of information from various sites in a specific industry.
I need to know what information they publish everyday.
Each site I am trying to scrape has its own format. But often have RSS feeds.

What type of tool can you guys suggest? Or do you know someone that can build one for me?

 

Y

yazzou

Registered Member

Dec 19, 2008

61

7

• Sep 1, 2009

• #2

screen scraper is the best for me. Got a server edition and cron, works with php, asp, and other languages

 

Djingel

Junior Member

Apr 9, 2007

150

110

• Sep 1, 2009

• #3

does anyone know if its possible to scrape more than one page, and let the script automatically recognize a page two?

 

H

hotcheats

Regular Member

Dec 2, 2008

326

75

• Sep 1, 2009

• #4

Check dapper.net or openkapow - I think they both can do that, but these are online tools.

 

MDSOperandi

Regular Member

Jul 10, 2009

230

212

• Sep 2, 2009

• #5

Also, on the off chance that you are going to get something built in .NET / C++ the HtmlAgility pack is awesome.

hxxp://htmlagilitypack.codeplex.com/

You can use XPath queries to parse HTML documents... I have made extensive use of it to scrape search engine results and the pages from those results.

Regards
MDS

 

B

boffmaster

Junior Member

Jun 1, 2009

144

32

• Sep 2, 2009

• #6

How about "google reader"

 

ukescuba

Senior Member

Feb 24, 2008

994

640

• Sep 2, 2009

• #7

this looks good - yazzou is this what your talking about???

Code:

http://www.screen-scraper.com/