Top C# (Headless) Web Automation Libraries and Toolkits?

[macdonjo3]

I'm just starting to learn C#. I'm a Python dev. Previously I'd use Selenium with PhantomJS for headless browsing.

I understand web requests in C# are pretty easy to do, but what options are there for headless browsing for a page that needs to execute javascript?

There is the IE WebBrowser object that you can place outside the frame dimensions. Is that really the most professional way to go about this in C#?

 

[sashablack]

lol so I see you decided to make your own?

-Sasha

 

[macdonjo3]

lol so I see you decided to make your own?

-Sasha

lol no, another project. deciding if i want to try a project in C# or not.

I ended up just hiring someone on SEO clerks to scrape a bunch of UIDs since it was a 1-time thing and it was faster to do that.

 

[kimmlouis]

Just use the default, WebBrowser Component.

 

[SlimProxies]

Awesomium, GeckoFX, CEF and other third party and open source wrappers. Most of them are webkit based (of course excluding GeckoFX). I personally like the API of Awesomium.

So, you have a bunch of choices for regular browsers, but not sure for headless ones, since I don't use headless ones.

 

[sam.hunt0710]

Awesomium, GeckoFX, CEF and other third party and open source wrappers. Most of them are webkit based (of course excluding GeckoFX). I personally like the API of Awesomium.

So, you have a bunch of choices for regular browsers, but not sure for headless ones, since I don't use headless ones.

Most of these leak memory if you are planning to do anything at webscale.

For small bot projects they should be ok, I like Awesomium over others, its a chrome wrapper in C# basically.

 

[Precubed]

Most of these leak memory if you are planning to do anything at webscale.

For small bot projects they should be ok, I like Awesomium over others, its a chrome wrapper in C# basically.

Flash leaks memory. The web browser components themselves do not leak memory.

 

[sam.hunt0710]

Flash leaks memory. The web browser components themselves do not leak memory.

OOO that reminds me flash is the shittiest piece of crap ever coded in the history of programming man kind (second place to IE ofc) and is still used so extensively all across the web.

Steve Jobs took a visionary move by disallowing it on iOS and fucking them over for the good of html5.

 

[Precubed]

So true. Flash is terrabad. So is Java. Both for so many reasons. And there was so much noise about Apple not including Flash support. The general opinion at the time was that Apple was crazy for not including flash support. That move effected the opinions and the direction of the entire industry.

 

[SlimProxies]

So true. Flash is terrabad. So is Java. Both for so many reasons. And there was so much noise about Apple not including Flash support. The general opinion at the time was that Apple was crazy for not including flash support. That move effected the opinions and the direction of the entire industry.

Woah woah. Java isn't bad. Java applets are bad. The JVM is very powerful, and the language is great as well.

 

[beatngu]

With C# there are tons of options,i like PhantomJS so why did you not use it in C# as well?
As far as Awsomnium i really hate that pice of junk,i have used the past year to migrate all my projects away from the shitty Awesomnium because some sites has found a way to block it.
The main problem was some sites was blocking awesomnium from uploading images and i simply found no way around it,with PhantomJS i managed to do it without any problem with injecting the site with a javascript.

Not sure if there is a fix for awesomnium now but to be honest i never looked back after i dumped awesomnium because there are many other options to use.
One example was uploading the images to ebay they where added but they never showed and the loading icon just keept on loading.

 

[rootjazz]

If you know selenium and phantomJS stick with those. The ghostdriver is included in selenium so controlling phantomjs is a piece of piss out the box