Best tool for scraping?

~@-kanye-west-@~

~@-kanye-west-@~

BANNED
Joined
Mar 12, 2009
Messages
581
Reaction score
246
I'm looking for a tool that spiders through a domain and exports lots of data that I can catch through regex and xpath selectors.

Can anyone suggest something that will work?

Thanks
 
Advertise on BHW
I have heard webharvy does that, no idea though. There must be a lot of those tools, you need search on Google.
 
If you are looking to scrape through xpath, i recommend you to try python selenium. It supports xpath and it has threading capability. It requires very low memory footprint so it is good for VPSs too. Let me know if you need a starting template. I have shared some in BHW (but single threaded ones.. Haven't shared any multi threaded python bot in bhw yet). I can link the thread if you need.

Scrapebox isn't that good when it comes to custom scraping according to what I have seen (I have scrapebox by the way.. if anybody has a tutorial for scrapebox using xpath, please point me to that. I would appreciate your help!!)
 
doepjohn said:
Have you tried Scrapebox already?
Click to expand...

Yeah, it's not good for what I want to do.

AuraMarketing said:
Webharvy can help you with that.
Click to expand...

I'll try that, thanks!

Gogol said:
If you are looking to scrape through xpath, i recommend you to try python selenium. It supports xpath and it has threading capability. It requires very low memory footprint so it is good for VPSs too. Let me know if you need a starting template. I have shared some in BHW (but single threaded ones.. Haven't shared any multi threaded python bot in bhw yet). I can link the thread if you need.

Scrapebox isn't that good when it comes to custom scraping according to what I have seen (I have scrapebox by the way.. if anybody has a tutorial for scrapebox using xpath, please point me to that. I would appreciate your help!!)
Click to expand...

Thanks for the recomendation. If webharvy doesn't work I'll try that!
 
So, you need to scrape the web – you're eyeing up that piece of data you'd like to extract and ... Netpeak Spider — scraping by Contains, RegExp, CSS Selector, and XPath
 
GSA email spider can do this job, but unfortunately this software is not being sold anymore.
 
Please try webharvy, I hope that will help you. Thanks
 
For curiosity: how many URLs to you intend to scrape? How many xpath-expressions aka fields to resolve?
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top