1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Limitations of C# Httwebrequests?

Discussion in 'C, C++, C#' started by Nick1, Sep 30, 2011.

  1. Nick1

    Nick1 Junior Member

    Joined:
    Oct 16, 2009
    Messages:
    196
    Likes Received:
    45
    I've been playing around with C# for a while, I've coded a few small bots with Httwebrequest.

    What are the limitations of Httwebrequest? It seems to have an ill reputation.

    I plan on using it for a while so I was wondering about any specific issues/bugs I should be aware of.

    Thanks BHW.
     
  2. Chris22

    Chris22 Regular Member

    Joined:
    Sep 29, 2010
    Messages:
    400
    Likes Received:
    1,059
    It's mostly down to performance I'd say, and using sockets gives you a better level of control. They are still pretty good though, and should be able to fulfill most of your requirements.
     
    • Thanks Thanks x 1
  3. darkmonk

    darkmonk Regular Member

    Joined:
    Nov 21, 2007
    Messages:
    226
    Likes Received:
    52
    I think people get confused as to what httpwebrequest is. It is the WebBrowser that is slow, but that sometimes you need to automate with a macro engine type of thing. Kind of how Magic Submitter used to work, it brings up a window framing the web browser and then simulates clicks and form field population.

    HttpWebRequest is just a wrapper around an underlying socket stream. It is using sockets already and it is quite easy to use proxies with it.
     
    • Thanks Thanks x 1
  4. smack

    smack Junior Member

    Joined:
    Feb 1, 2010
    Messages:
    182
    Likes Received:
    78
    Occupation:
    Software Engineer/Evil Genius
    Location:
    inside .NET
    httprequest is slow itself when compared to a good custom implementation of the sockets.

    it is a very poor wrapper and is buggy, bloated, and tends to mishandle cookies.
     
    • Thanks Thanks x 3
  5. Nick1

    Nick1 Junior Member

    Joined:
    Oct 16, 2009
    Messages:
    196
    Likes Received:
    45
    That was my experience also. I was suspecting that it was the cookies but I never looked too much into it. Thanks for confirming.
     
  6. scriptomania

    scriptomania Junior Member

    Joined:
    Dec 28, 2010
    Messages:
    127
    Likes Received:
    249
    Occupation:
    A full time pirate at sea
    Location:
    The European capital of politics
    Limitations you say? No SOCKS4/5 for starters...
     
    • Thanks Thanks x 1
  7. something77

    something77 Registered Member

    Joined:
    Jan 6, 2010
    Messages:
    73
    Likes Received:
    10
    I have read up quite abit recently about sockets vs httpwebrequests. However,( and I have edited your quote to emphasise the word "good") to make a good implementation that would replicate everything httpwebrequests can do would be quite time consuming and complicated (unless you know what you are doing, but assuming this question is being asked it is assumed you do not) and depending on what you want to do, imo not worth it.

    If you are coding an app to sell, then perhaps it might be worth it, if you are just writing custom spam tools, then I don't think it is worth it for the extra performance you can squeeze out it.

    If you are using c# to start with, you are not too concerned with performance anyway.
    If you want performance go c++ and use sockets.
    If you want speed of coding and getting projects complete, c# and httpwebrequests.


    With regards to the cookies, are you sure and have back up evidence of this. I have never heard this mentioned before. When I have blamed something for a failure, it has been caused by a bug in my code, not the httpwebrequest.
     
    • Thanks Thanks x 1
    Last edited: Oct 3, 2011
  8. xenon2010

    xenon2010 Regular Member

    Joined:
    Apr 27, 2010
    Messages:
    231
    Likes Received:
    48
    Occupation:
    web and desktop apps programmer
    Location:
    prison
    Home Page:
    nobody use socks anymore...
    unless if you have LOTS of FREE time and you wanna do everything by yourself..
    In that case why you are bothering with C# in the first place. you should use C++/C instead.
    IMO, I don't see httpwebrequest has any serious problems (maybe a bit with the performance) thats all.
     
  9. something77

    something77 Registered Member

    Joined:
    Jan 6, 2010
    Messages:
    73
    Likes Received:
    10
    Does anyway have any actual figures / benchmarks for speed improvements for sockets vs webrequests?

    Does the problem only occur with 100+ threads?
    One webrequest call vs one socket call, speed dif is 0.000001secs for sockets
    etc
     
  10. smack

    smack Junior Member

    Joined:
    Feb 1, 2010
    Messages:
    182
    Likes Received:
    78
    Occupation:
    Software Engineer/Evil Genius
    Location:
    inside .NET
    in regards to socket performance unless you're really going deep and need every ounce of overhead removed, say for a high throughput atomic mailer, then C++ vs. C# isn't going to be an appreciable difference in most cases.

    in my opinion it is well worth doing your own implementation of the sockets as opposed to relying on the piss poor functionality provided through httpwebrequest. you only progress your development skills by pushing the envelope and venturing in to uncharted territory. my dev kung fu has benefited an untold amount by not being bound and constrained to the things others have implemented.

    in regards to the cookies in httpwebrequest, myself and many others (search) have encountered issues with the cookie handling in httpwebrequest that were verified not to be an issue with the code around it, but with the internal classes itself. if you really think about the implementation that MS provided of httpwebrequest it's not actually meant for the heavy lifting of many BH applications.

    doing your own implementation will pay dividends not only in the areas of performance, reliability, and functionality, but will also give you insight in to deeper levels of control and program design. something any developer can benefit from.

    but then again, that's me. i have never been a fan of "good enough".
     
  11. something77

    something77 Registered Member

    Joined:
    Jan 6, 2010
    Messages:
    73
    Likes Received:
    10
    If I have my developer hat on, then definiately I agree with you. And I can sympathesise with not being a fan of "good enough".

    But then I put on my "making money hat" and I come up with a completely different answer.

    I started a couple of years back developing personal tools, trying to get them working perfectly from the user perspective and knowing the code was damn good. Problem was, as I learnt more I keep improving and improving and never using.

    This time round and getting things "good enough" and then using them and making lots more ££££ than before. Although have spent all morning improving code that already works. Using a measure of money earned and time spent, this morning was a complete waste of time.
     
    • Thanks Thanks x 1
  12. smack

    smack Junior Member

    Joined:
    Feb 1, 2010
    Messages:
    182
    Likes Received:
    78
    Occupation:
    Software Engineer/Evil Genius
    Location:
    inside .NET
    the developer hat and the money making hat don't need to be mutually exclusive.

    i look at this way:

    you build something more stable, efficient, and reliable than httpwebrequest. that means your programs built on it will be better, run longer, have less issues. that's a revenue increase right there.

    in addition you can implement things on your own that will save time and save code in the long run, another win.

    it's an investment, but one that will pay off in both having a better implementation and having more practice and better knowledge that will open more doors for you in the future.

    i can say with 100% certainty that without the custom http class i use i would not have had nearly the monetary returns or skills set that i have today.
     
  13. Monrox

    Monrox Power Member

    Joined:
    Apr 9, 2010
    Messages:
    615
    Likes Received:
    579
    The speed is a bad criterion. It rather depends on the bandwidth of the internet connection, than the choosing a socket over a webrequest implementation. Unless it is some ancient Celeron.

    For me the biggest issue with the webrequest has always been the timeouts. They are not flexible enough and a not responding webserver has the potential of locking a program pretty hard. There is no easy way to simply break operation after x seconds unresponsiveness.

    But it is not a language problem too. Normal browsers from every flavor of MS, FF and Chrome still hang unrecoverably here and there occasionally.
     
    • Thanks Thanks x 1
  14. frozenocean

    frozenocean Junior Member

    Joined:
    Sep 30, 2011
    Messages:
    100
    Likes Received:
    11
    Location:
    both arctic
    What is the problem with httpwebrequest!!!

    No problem at all...

    The problem is specific to you to define it. When you only you define a problem that is a problem specific to you.

    httpwebrequest is designed and developed to do some tasks in a certain way. You dont like this things after knowing it well dont use it. but You think is it worth debating based on your idea not the problem. You will code a socket to over httpwebrequest for few simple tasks!!! Or will it give a certain amount of flexibility with reliability!!! The question is of worthiness of doing some thing. certainly for a certain programmer the more line he writes the more his time consumed(with thought). Does that all makes sense to your problem!!! That is the answer...not a second slow or first. freedom or not freedom. Let say i need to make a post to some site with some params. The least possible code with reliability I will look for. If I can do that with few line of code why I code sixteen line...
     
    • Thanks Thanks x 1