Site Scraper
- Thread starter addbravo
- Start date
blackhatchampion
Senior Member
- Dec 29, 2010
- 1,069
- 552
I am looking same here.
christoss1959
Senior Member
- Nov 25, 2010
- 896
- 1,176
Do you want a software than can scrap a complete website?
Or do you want a software that can scrap the text of a website?
Or do you want a software that can scrap the text of a website?
kokoloko75
Elite Member
- Jan 1, 2011
- 1,627
- 1,950
HTTrack :
Beny
Code:
http://www.httrack.com/Beny
searchrecognition
Newbie
- May 5, 2011
- 3
- 0
try irobot - and if your a programmer you can get it to do loads ?
there is no tool out there to scrape any site 
... you need a custom bot ... you may find (or make) a bot that will scrape all text from a site but there is no tool that will scrape onlt the content of an article or a description from any site out there... Sites are different and you need to instruct your bot where to find (and how to find) the text in a page
... you need a custom bot ... you may find (or make) a bot that will scrape all text from a site but there is no tool that will scrape onlt the content of an article or a description from any site out there... Sites are different and you need to instruct your bot where to find (and how to find) the text in a page
jkwilson78
Regular Member
- Jun 24, 2010
- 224
- 313
Like others have said, there is no tool that can scrape any site from out of the box. But there are tool you can use to scrape practically any site you just have to configure the software to do it.
One of the easiest ones I have used (and the cheapest for what it can do) is web content extractor.
http://www.newprosoft.com/web-content-extractor.htm
They have a version that rotates proxies you just have to ask them for it and their support is awesome. There have been a few sites I've had trouble scraping and after contacting support with the details they have set up the scraper for me and even customized the software itself to accommodate the site I was trying to scrape.
One of the easiest ones I have used (and the cheapest for what it can do) is web content extractor.
http://www.newprosoft.com/web-content-extractor.htm
They have a version that rotates proxies you just have to ask them for it and their support is awesome. There have been a few sites I've had trouble scraping and after contacting support with the details they have set up the scraper for me and even customized the software itself to accommodate the site I was trying to scrape.
paulblack2025
Newbie
- Jul 8, 2016
- 15
- 1
Any site??
I'm using Octoparse | Web Scraping Software.
But I don't think it will work for any sites.
Writing a scraper is kinda hard for me.
I'm using Octoparse | Web Scraping Software.
But I don't think it will work for any sites.
Writing a scraper is kinda hard for me.
Tatterdemalion
Newbie
- Jan 8, 2016
- 39
- 5
hide me
- Jul 8, 2010
- 3,994
- 823
That's the beauty of scraping you mostly need different scripting for each sites which are big sites like fb,G,Yt etc due to their different structures,securities etc.
And for small sites using Wp,Joomla,etc mostly can be scrapped with once code and scrapping method normally.
And for small sites using Wp,Joomla,etc mostly can be scrapped with once code and scrapping method normally.