What programming language(s) to learn for building advanced web scrappers?

I'd been an SEO guy for years now. And over time I've headed more towards building my niche blog network. It's going well so far, as long as ROI is concerned.

As a part of my working methods, I've got to analyze tons of KW data, content pages and sort them out in some way that is presumably different than what many other people do. And it works.

Now, I'm looking forward to building a customized tool in this regard to shorten up the effort and cut the clutter. To be specific, I want to scrap webs, collect data, sort/filter them out, and organize those in an actionable manner(to make it easy for writers who'll take over from there).

And obviously, I've got to learn programming languages for that. Right now, I have nothing but some C and Fortran basics.

So, what are the languages that I should learn in the first place? It would be better if you guys can draw a complete image of what after what I should step into.

Thanks in advance.

If you have a relative knowledge of c you should try zenno poster. I've been using it for years and it's great for creating your own custom tools for yourself. You can try ubot too but it's more expensive and it's in python. Zenno added a monthy license fee if you want to try it, they also have demo but it doesnt have proxy support.

depends on the your size and scale and the types of websites you're scraping.

if you're building a huge webscraper crawler in order to try and scrape crawl most of the web (think ahrefs) then building it in a compiled language (C, C++, Rust, Go even Zig maybe).

If you're dealing with sites with js front ends and your not doing a lot of stuff then learning a bit of JS and then using puppeteer is probably the easiest solution.

If you want to build for windows, I would go with c# it have all the tools out there and is easier than C
If you speed and multi platform go with python

net307 said:

If you want to build for windows, I would go with c# it have all the tools out there and is easier than C
If you speed and multi platform go with python

Click to expand...

Speed and Python???

Like a saas/web tool? I'd go with a classic PHP/JS stack.

PHP with something like symfony or Laravel in the backend (PHP is really great at data manipulation) and JavaScript with React or Angular in the front end. Some might say that nodeJS would be better for the backend but I personally don't find JavaScript to be a very scalable language. I'd choose PHP or even python before JavaScript for the server-side.