Direction needed. I want to build a Semi-automatic Web browser in C#

marcelomusza

Junior Member
Joined
Oct 18, 2012
Messages
198
Reaction score
61
Hi guys,

I am a Web Developer, so I have solid knowledge in C#.
As I am working in my IM business like crazy, I want to create a semi-automated software to use several accounts to view and comment YT Videos manually, just by hand, selecting users from a list.

I know how I am going to build this piece of software, but I never made this kind of tools before. I would like to hear your suggestions about where to start, is there a manual or tutorial around to educate myself about how to create a web browser kind of software?
I am googling this but I am finding very random approaches, and I come here because I am sure you can easily share with me a more targeted approach.

Should I use the WebBrowser class for this?

Thanks in advance! every piece of advice will be much appreciated!
 
Simplest is to use WebBrowser. It has multithreaded issues and also can sometimes be fooled as to when the page has actually loaded but it's the simplest to get going with.
 
I dont know about views, but commenting you can make with Youtube API.
 
gud luck ! Hope you do share your bot with BHW .. !
 
WebBrowser.

Just whatever you do, do it slow, make it act like a real human. YT keep changing things around and ever the real comments getting marked as a spam, I'm not even talking about captcha on the comments.

Good luck mate
 
Use WatiN. You can tie it to a WebBrowser control as well.
 
Webbrowser control is better, you can use WebClient or HttpWebResponse but it is not easy to post data.
Webbrowser is slowest but it is not different too much from Internet Explorer.
 
The WebBrowser controll is essentially Internet Explorer wrapped up.
 
There is a big problem with the WebBrowser: bad proxy support & no file upload :/
 
Forget about WebBrowser, WatiN
First approach is to use WebClient, you must inherit the class to add additional functionality
Code:
    using System;
    using System.Net;
    using System.Net.Security;
    using System.Security.Cryptography.X509Certificates;
    public class WebClientEx : WebClient
    {
        static WebClientEx()
        {
            ServicePointManager.Expect100Continue = false;
            //Ignore certificate errors
            ServicePointManager.ServerCertificateValidationCallback = (RemoteCertificateValidationCallback)((object sender,
            X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) => true);
            //This line is very important for multithreading
            //by default you can have up to 2 simultaneous connections, no matter how many threads you are creating
            ServicePointManager.DefaultConnectionLimit = 1000;
        }


        public WebClientEx()
            : base()
        {
            this.AllowAutoRedirect = true;
            this.Cookies = new CookieContainer();
        }
        
        protected override WebRequest GetWebRequest(Uri address)
        {
            //You can add other headers too
            HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
            if (request.Method == "POST")
            {
                request.ContentType = "application/x-www-form-urlencoded";
            }
            //the webclient will handle GZip and Defalte compressions automatically, improves the performance a lot
            request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
            request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12";
            request.CookieContainer = this.Cookies;
            request.Timeout = 10000;
            return request;
        }
        
        //Add cookie support
        public CookieContainer Cookies { get; set; }
    }
Basically WebClient is wrapping WebRequest and WebResponse classes and simplifies the things.
Code:
WebClientEx web = new WebClientEx();
string htmlGet = web.DownloadString("yoururl");
string htmlPost = web.UploadString("yoururl", "yourpostdata");
byte[] dataGet = web.DownloadData("yoururl");
byte[] dataPost = web.UploadData("yoururl", Encoding.UTF8.GetBytes("yourpostdata"));
However you have no multipart-formdata support.

The second approach is to use HttpClient, which is better and have multipart-formdata support. It is in .NET 4.5, but you can use it in VS 2010 and .NET 4.0
Get it from NuGet:
Code:
PM> Install-Package Microsoft.Net.Http
However, the latest version of HttpClient has no synchronous methods.
I am using the older version with the sync methods, i am not sure why MS forced us to use only async methods.
Get it from NuGet:
Code:
PM> Install-Package HttpClient -Version 0.3.0
Or
Code:
PM> Install-Package HttpClient -Version 0.5.0
- this is the lastest version that still has sync methods.

Fiddler is a very helpful tool to intercept and monitor HTTP traffic, it is also free. Google it, i cannot post links.
 
Last edited:
Doing anything with the WebClient object is a PITA, I think for complex bots you need another tool, another environment. I have sold bots using both techniques and WebBrowser is crappier but easier, WebClient is much more complete, but very lengthy to do something with it.
 
Last edited:
Use Selenium (http://seleniumhq.org/) it is completely free / open-source, depending on your needs you can use their embedded browser, Firefox, Chrome (and I believe IE *shudder*). There are .NET bindings and its easy to use from C#. You have full DOM access to the loaded document, you can inject JavaScript instructions to the pages to simplify automation instead of trying to navigate the DOM and click links/buttons.

I'm using the Java bindings with Selenium for all my IM bots, I can deploy them on cheap linux VPS's for a few cents/hour.
 
Can you upload files with Selenium, into the filefield? That will be the full solution if is possible
 
Any browser automation is slow as hell and it is not easier coding a bot with browser than with just requests.
And how about mutlithreading ? You are going to create hundreds of browser objects, sure this is a good idea.
I hope the client will be satisfied with a program that consumes few GBs of RAM.
 
You can comment with Youtube API. Google has SDK for .NET. But make sure that you are using proxies and rotate several api keys. Or you get ban.
 
Webclient is one way to do it, however you have to realize that a wc won't process javascript for you. Sites that really heavily on jscript won't work if you try to drive it using low level post/get.

Selenium is a good way to go, however its not headless. IE a browser window is always open.
 
Thanks Ikstob, I've been trying to play with Selenium to replace a current Firefox/iMacros based setup.
I hope it will prove leaner than my existing setup.


Use Selenium (URL REMOVED) it is completely free / open-source, depending on your needs you can use their embedded browser, Firefox, Chrome (and I believe IE *shudder*). There are .NET bindings and its easy to use from C#. You have full DOM access to the loaded document, you can inject JavaScript instructions to the pages to simplify automation instead of trying to navigate the DOM and click links/buttons.

I'm using the Java bindings with Selenium for all my IM bots, I can deploy them on cheap linux VPS's for a few cents/hour.
 
Back
Top