Best Software for Web Scraping (Discussion)

[Donbuffy]

Hello Guys, trying to go into proper data mining for my new project and i've been surfing the net for a long while, i've read a lot of reviews and i've seen a couple of sales thread on here but i'll need to use this thread as a basis of discussion cos it seems am not the only one asking of this information, it should help other noobs out there too, if you've been using a web scraper automation software please comment on this thread, it would enable one make a choice on the best.

What's the best leads generation software you've used?
If you don't use a software, what script/plugin do you use to extract tons of leads?

 

[letuvis]

I was using Parse Hub as my software to scrape data. It's easy to use and has a free plan. Would advise to use Parse Hub it's the best imo.

 

[Peter Samaki]

I was using Parse Hub as my software to scrape data. It's easy to use and has a free plan. Would advise to use Parse Hub it's the best imo.

Limited features are free.. But expensive for normal meaningful scraping.

 

[KJREDDY247@]

1. web scraper
2. parse hub
3. scrapy

I use these because I don't have proper knowledge in coding
if you know them well;
you can go with
python with selenium, beautiful soap etc

 

[letuvis]

Limited features are free.. But expensive for normal meaningful scraping.

Of course you get limited features when you use free plan for the app. This is normal from my point of view. I used free plan of Parse Hub and everything was scraped for me like a charm. It gave me everything I need, so I don't see any big issue regarding this..

 

[Deleted member 657006]

piaghim bot it the best for any leads .....
(( cost not free))

you can also use scrapebox , does many things .(( cost not free))

you can also use gsa tools they do many scrapping tools. (( cost not free ))

all the above are paid tools but have free trails .

good luck.

 

[Donbuffy]

piaghim bot it the best for any leads .....
(( cost not free))

you can also use scrapebox , does many things .(( cost not free))

you can also use gsa tools they do many scrapping tools. (( cost not free ))

all the above are paid tools but have free trails .

good luck.

Heard Scrapebox is the best, i'll have a look at paigham bot

 

[nirose]

beautiful soap etc

beautiful soup*

On topic, scrapping is the easiest thing you can do in programming. and there are a ton of tutorials everywhere. From my experience in JavaScript, PHP, and Python scraping couldn't be any easier. you just have to learn some CSS selectors or XPath and you can grab a lot of data on auto. You can go to lengths with header modification and authentication but most of the sites don't require this.

The only problem starting with your own scraping bot with going to be threading, as single threads are going to be slow. but it should be a learning process. If you can pay, scrapebox is a good investment. it's really fast

 

[KJREDDY247@]

beautiful soup*

My Grammarly is kind of crazy some times , Yeah I am planning to learn Python to do basic scrapping at least

 

[Gogol]

There's no "best" software. It's about what tool you want to use to scrape. I personally prefer python requests (if there's no need for js support) or selenium (if the bot has to be js enabled).
Perl can also do the same job, as it has full selenium support. Even PHP can do the same job, but it will be tougher to do it in php; if js support is needed. I am not sure if php has selenium support.

As for scrapebox, I am not actually sure. I do have sb licence, but I seldom use it. I mostly code my own bot whenever needed.

 

[Donbuffy]

beautiful soup*

On topic, scrapping is the easiest thing you can do in programming. and there are a ton of tutorials everywhere. From my experience in JavaScript, PHP, and Python scraping couldn't be any easier. you just have to learn some CSS selectors or XPath and you can grab a lot of data on auto. You can go to lengths with header modification and authentication but most of the sites don't require this.

The only problem starting with your own scraping bot with going to be threading, as single threads are going to be slow. but it should be a learning process. If you can pay, scrapebox is a good investment. it's really fast

There's no "best" software. It's about what tool you want to use to scrape. I personally prefer python requests (if there's no need for js support) or selenium (if the bot has to be js enabled).
Perl can also do the same job, as it has full selenium support. Even PHP can do the same job, but it will be tougher to do it in php; if js support is needed. I am not sure if php has selenium support.

As for scrapebox, I am not actually sure. I do have sb licence, but I seldom use it. I mostly code my own bot whenever needed.

So from all am understanding here, there's no best bot it all balls down to what is to be scraped and how the user utilizes the tool... so it's best one learn python to build a custom tool that enables one to scrape, right? So computer language for scraping is Python, JavaScript and PHP?

I know many Digital Marketers who has zero programming knowledge but use tools to scrape leads, i wonder what tool they use.

 

[Gogol]

I know many Digital Marketers who has zero programming knowledge but use tools to scrape leads, i wonder what tool they use.

The problem with a scraper is that it is pretty much platform dependent. A scraper which works on platform A, might not work on platform B. It really does depend on what actually you are trying to scrape. In case you are interested in Scrapebox, here's a list of available plugins that are free with SB (so that you can decide whether to try SB or not):





Believe me, making a scraper with python isn't as hard as you think it is. With this little skill, you will not be tied to a specific tool or platform.

 

[javadth]

scrapebox i am using it

 

[710fla]

Scrapebox hands down. Lots of extra features for free too

 

[ebubekirtrkr]

Python Selenium

 

[BHSEONewb]

The problem with a scraper is that it is pretty much platform dependent. A scraper which works on platform A, might not work on platform B. It really does depend on what actually you are trying to scrape. In case you are interested in Scrapebox, here's a list of available plugins that are free with SB (so that you can decide whether to try SB or not):

View attachment 119313



Believe me, making a scraper with python isn't as hard as you think it is. With this little skill, you will not be tied to a specific tool or platform.

Is there a tutorial you recommend for learning this? It's been on my list to learn python and web scraping for quite some time now, but finding the right course (and time) has been the main problem.

 

[Gogol]

Is there a tutorial you recommend for learning this? It's been on my list to learn python and web scraping for quite some time now, but finding the right course (and time) has been the main problem.

Good question. Yepp I have a few channels on youtube that I really enjoy learning from (you can never learn enuf lol).
As for example:

https://www.youtube.com/channel/UCWv7vMbMWH4-V0ZXdmDpPBA
https://www.youtube.com/user/edurekaIN   (this is an Indian channel, and you might not like the pronunciation. Nevertheless, the amount of knowledge shared on this channel is freaking awesome. Just get used to the pronunciation lol).
Also, 
https://www.youtube.com/user/schafer5

I have a few more in my mind, but can't find them right now. I will update this thread later.

 

[BHSEONewb]

Good question. Yepp I have a few channels on youtube that I really enjoy learning from (you can never learn enuf lol).
As for example:

https://www.youtube.com/channel/UCWv7vMbMWH4-V0ZXdmDpPBA
https://www.youtube.com/user/edurekaIN   (this is an Indian channel, and you might not like the pronunciation. Nevertheless, the amount of knowledge shared on this channel is freaking awesome. Just get used to the pronunciation lol).
Also,
https://www.youtube.com/user/schafer5

I have a few more in my mind, but can't find them right now. I will update this thread later.

THANK YOU!!!!

 

[Donbuffy]

Good question. Yepp I have a few channels on youtube that I really enjoy learning from (you can never learn enuf lol).
As for example:

https://www.youtube.com/channel/UCWv7vMbMWH4-V0ZXdmDpPBA
https://www.youtube.com/user/edurekaIN   (this is an Indian channel, and you might not like the pronunciation. Nevertheless, the amount of knowledge shared on this channel is freaking awesome. Just get used to the pronunciation lol).
Also,
https://www.youtube.com/user/schafer5

I have a few more in my mind, but can't find them right now. I will update this thread later.

Oh wow! Thanks

 

[Advertising Manager]

Good to know for future scrapping