[C#] HttpWebRequest/Socket Sync vs. Async

Discussion in 'C, C++, C#' started by Stalli0n, Mar 19, 2011.

  1. Stalli0n

    Stalli0n Junior Member

    Nov 17, 2010
    Likes Received:
    Hey, i do a lot of automating stuff on the internet and currently i'm using HttpWebRequest.

    But with multithreading it sometimes just freezes, no timeout.

    So i looked at the asynchronous methods of HttpWebRequest and found out that there are also async methods for Sockets. I wanted to switch from HttpWebRequest to Sockets for a long time but was to lazy to implement Http for Sockets.

    And i read that i can't use async Socket methods with Https.
    I also want SOCKS4/5 support.

    So what would be the best solution?

    Write a class with Sockets that switches from async to sync methods when there's a Https site?
    Implement SSL myself? :eek:
  2. reinrein

    reinrein Regular Member

    Feb 8, 2008
    Likes Received:
    if you want to support https with socks proxy, it will be headache. if you decided to do so, proxy communication uses bigendian for paramater lengths, while .net uses littleendian, I just told you this because this was one of the part I spent so much time with(I didnt knew about it).

    Also one of the hard parts is supporting encoding, "gzip and deflate". If you use sockets, you will have to do the decoding yourself, which is hard because many sites doesnt follow standards, like yahoo.

    If youre only problem with httpwebrequest is freezing, then its not a reason to go lower down to sockets. But heres the advantages of of sockets over httpwebrequest:

    1. Full control of headers, httpwebrequest leaves footprints like "connection: closed" which doesnt jive with request headers of normal browsers(usually "connection: keep-alive")

    2. You are not limited to the environment of httpwebrequest, which I think you already noticed, have bugs. For instance, the cooki class have errors. Also limited to http proxies only, and the proxy handling itself have bugs also. Same with https/ssl support, there are bugs like sending headers twice etc.

    3. You can create your own class to fully support new http standards using sockets.
    • Thanks Thanks x 1
  3. hiderightnow

    hiderightnow Junior Member

    Jul 19, 2010
    Likes Received:
    You could look at the chillkat library if you do a lot of web automation, it's a great tool to use under any circumstances (not necessarily only web related).
  4. crepito

    crepito Junior Member

    Oct 5, 2008
    Likes Received:
    Hello, just want to point out that you could create async httpwebrequests with timeout as you probably already know by now. easier than using sockets. Not so powerful tho.