C# Vs Python For Advanced Web Scraping?

Python takes care of alot of stuff for you, so less work which means high productivity.

c# because compiled languages are just better. The end.














IMO of course :-D

This statement hurts... BTW C# gets *compiled* to msil which is then run on an *interpretor* on the target machine.

In python you have really good framework for scraping - scrapy

if you plan to just use it for yourself and just need it for easy tasks like scraping and parsing html/forms + want results fast then go with python.
however if you take the time now and learn c#, you will be able to code much more advanced types of programs later on.

HTML, JavaScript, CSS and HTTP requests first. Regex also. The other is same for both languages! For using HTTP requests you will need to know Regex. For using browser automation, you will need to know CSS, HTML and JavaScript. The both C# and Pyton are objected oriented - you can make functions/methods, that can be called from one row late. I love PHP + HTTP requests + regex + cronjob - small files, fast work on total autopilot, but needs to dig deeper at sometimes (because the HTTP requests) - just curl.

My personal recommendation - learn HTTP requests and Regex in case you want to learn. The real twist here is HTTP VS Browser automation. Nowdays the major sites becomes more and more protected for the both scrapping ways. HTTP handles code returned from the server and very regular is protected from some JavaScript generated token/s which also needs to be reproduced..... Browser automation works after the browser handle all javascripts and gets the content, but the content becomes more and more dynamic - as example on CSS classes and id's of the HTML elements on Twitter are generated with random names (I sow this on HMA for the very first time) reducing the footprints to minimum...

black_hat_newbie said:

hi guys,
I want to be able to scrape google maps, google places, amazon, facebook and other websites like yellowpages, yelp, realtor.com, etc. and save the data in an excel sheet or database.

Which would be a better one for me to learn? or both would be same and it wont matter?

Click to expand...

Python.