youbike
BANNED
- Mar 14, 2016
- 71
- 73
it depends , baby .. if you iq is high , use manual web because it could be so fast if you know what you doing .. if your iq is medium to low get software solution but this can be contain nagware , ransomware , adsware , hideware , captureware , spamware , buymeware , marrymeware , addmepleaseware , joinmeware
Jinyus
Regular Member
- Jul 6, 2017
- 416
- 191
If you know how to scrape with python and use databases. Then you dont need software solutions. It also depends on your use case...If you just want to extra data and store it somewhere or put on your site, manual scraping is better. If you want to submit contact forms, some software solutions will have detection systems for various cms etc.... but it's nothing that you can't do.
If you state your skill level and your intent, I can advise you more effectively.
If you state your skill level and your intent, I can advise you more effectively.
websitegeek
Junior Member
- Dec 22, 2011
- 103
- 15
Python is great for scraping. You can create software solutions with Python. So really the answer to your question is Python is the raw material that you can use to build a software solution. It can't really be compared.
Vamsi9002
Newbie
- Feb 1, 2019
- 43
- 8
@youbike @Jinyus @websitegeek Hey thank you for your replies. I am a newbie and I have very limited knowledge with manual web scraping and from what was mentioned above, I do realize that there is more functionality with it. I am a digital marketer and my goal is to reach out to the people after I data mine. So a software solution will save me time but it costs me money.
@websitegeek I have been using the ParseHub free trial and it is exactly what I want. It is a little pricey at $150+ a month and that is why I am exploring manual options. What do you think is the best route for me as I am a novice at webscraping?
@websitegeek I have been using the ParseHub free trial and it is exactly what I want. It is a little pricey at $150+ a month and that is why I am exploring manual options. What do you think is the best route for me as I am a novice at webscraping?
trunkss
Registered Member
- Aug 9, 2010
- 93
- 51
If it's something quick and easy you can use the various free plugins, software and websites.
As things get more complicated you probably won't be covered with these ready solutions and you'll have to write your own bots.
If you already know html and javascript then with a little bit of studying you can start making your own scrapers, if you know Jquery then you are more than ready to start.
Once you get going and you take on more big projeects you'll need more resources, you pc might not be enough so you you'll have to use services or build your own servers. Anyway, tha'ts a long time ahead so you shouldn't worry about it.
As things get more complicated you probably won't be covered with these ready solutions and you'll have to write your own bots.
If you already know html and javascript then with a little bit of studying you can start making your own scrapers, if you know Jquery then you are more than ready to start.
Once you get going and you take on more big projeects you'll need more resources, you pc might not be enough so you you'll have to use services or build your own servers. Anyway, tha'ts a long time ahead so you shouldn't worry about it.
whtflcn
Newbie
- Nov 19, 2019
- 16
- 1
I'm a little late to the party, but the nice thing about Python is its great for making stuff relatively quickly. The syntax isn't verbose, and there are a whole lot of open sources libraries you could easily utilize, including some that do scraping. A paid/commercial service might be all that you need, but the nice thing about having something custom made (either by you or someone else) may not only be a far more economical over the long term, but could have functionality that commercial options don't/won't implement.
- Jul 26, 2015
- 2,625
- 712
It's not clear in a few words...you can easily read this post to help you out,
https://www.privateproxyreviews.com/web-scraping-python-scraper-tools/
In short,
Python Web Scraping, which offers more freedom, more personalization, and more customization.
Other web scrapers, that can save your time, for those services offer lots of "templates"...amazon, youtube, Reddit, lots of projects are the same, you can easily call to use...
https://www.privateproxyreviews.com/web-scraping-python-scraper-tools/
In short,
Python Web Scraping, which offers more freedom, more personalization, and more customization.
Other web scrapers, that can save your time, for those services offer lots of "templates"...amazon, youtube, Reddit, lots of projects are the same, you can easily call to use...
- Mar 30, 2010
- 1,279
- 415
You must be good at Python and web scraping!Python. all pros, no cons
PIEGE
Newbie
- Jul 4, 2019
- 35
- 7
I am only starting out, but more and more python is the tool i reach for. the language is easy for my brain to think about, plus features for everything. bonus it is native to linux so any code i write automatically close to the servers i work on. can even run javascript engine 'inside' python script using libraries.
you will find people that will argue against python being the right tool for the right job... pay them no mind.
RockTheStreet
Senior Member
- Jul 16, 2019
- 1,148
- 129
For me, only manual cleaning.
D
Deleted member 657006
Guest
whatever language your doing , goto a forum that dedicated for that language , and I promise you that there be so many examples of code of anything you can think of, your soon come very familiar with the in and outs, you can master any language with copy and paste but if it your career need to study it hard..
it so true copy and paste wannabe programmers do survive, make real hardcore programmers mad lol lol.
there so many ppl doing python that their thousands of grate working examples in python forums that are free to join.
I am going to get cursed now programmers hate copy paste wannabes
to make it even easier you can get a paid ide program that helps you code , most programmers use notepad or notepad ++ but there real powerfull ones that nearly code for you.....
now I am going to get a black eye...lol
it so true copy and paste wannabe programmers do survive, make real hardcore programmers mad lol lol.
there so many ppl doing python that their thousands of grate working examples in python forums that are free to join.
I am going to get cursed now programmers hate copy paste wannabes
to make it even easier you can get a paid ide program that helps you code , most programmers use notepad or notepad ++ but there real powerfull ones that nearly code for you.....
now I am going to get a black eye...lol
Last edited by a moderator:
D
Deleted member 657006
Guest
I totally agree but learning only reading and taking a go to see if it true, but to master something takes much skill." I am going to get cursed now programmers hate copy paste wannabes "
some people claim to 'know' a language after 3 months of copy paste, youtube and BHW questions
marketing is only traffic once you understand that everything falls into place.
i love reading every subject and method sometimes even print them to deeply understand I am very obsessed with marketing.
Last edited by a moderator:
- Oct 24, 2011
- 3,222
- 3,626
Mikall
Newbie
- May 15, 2020
- 25
- 12
It is really easy to do web scraping using python, you may watch several tutorials and you will be ready to build your own scraper which will be fast and free. As for software solutions, first of all, it is hard to find something good and free of charge. Secondly, if you find it, it will be full of ads and viruses.