TheGateKeeper
Regular Member
- Nov 9, 2011
- 226
- 58
In c# that is. Need to ask some questions and posting to websites using javascript validation.
Contact me via pm please.
Contact me via pm please.
Anyone with experience using the WebBrowser control?
- Thread starter TheGateKeeper
- Start date
DigitalChuck
Newbie
- Apr 30, 2009
- 39
- 3
I've got a lot of years of exp working with the WebBrowser control, but sorry mate not in c# - i'm more of a VB guy.
Have you tried the Webkit control yet? I've read somewhere that Javascript was easier
http://www.webkit.org/
http://sourceforge.net/projects/webkitdotnet/
sorry i couldn't help more
Have you tried the Webkit control yet? I've read somewhere that Javascript was easier
http://www.webkit.org/
http://sourceforge.net/projects/webkitdotnet/
sorry i couldn't help more
TheGateKeeper
Regular Member
- Nov 9, 2011
- 226
- 58
Maybe you can help me, sent pm.
smack
Junior Member
- Feb 1, 2010
- 182
- 80
it's the same class and same control provided by the framework, the syntax looks a little different but that's about it.
that being said i would advise anyone to avoid using the web browser control at all costs. it is slow, buggy, error prone, and bloated. you can't use multithreading with it and it will only accept one proxy at a time.
if it's just JavaScript validation you're worried about most times you can just ignore the client side validation when doing your submits. as long as you know the values you're submitting are correct they should be able to be passed to the server with no issues. the only time this can become a problem is if they're using some sort of specific JS generated values that are required for the submit. even in that case though you can usually calculate those values yourself in your C# code.
Chris22
Regular Member
- Sep 29, 2010
- 400
- 1,069
Do you know if that one proxy per time is process specific or operating system wide? For example lets say I made a bot that uses a web browser control with a proxy, and already had a different piece of software running that is also using a browser control and a proxy, are they both using their own proxies or is the proxy used determined on which webbrowser control set the proxy last?
TheGateKeeper
Regular Member
- Nov 9, 2011
- 226
- 58
I think it creates an instance for each WebBrowser control.
smack
Junior Member
- Feb 1, 2010
- 182
- 80
as far as i am aware it is operating system wide since it actually uses the Internet Options setting for a proxy server, same as you would access through IE's Tools menu.
Wolfpack
Junior Member
- Jul 13, 2011
- 166
- 351
This this this! What specifically are you trying to do? It might help us to see examples, and maybe I could help guide you to some articles that might help you do this with httpwebrequest or socks, as they are much much better for using multiple IPs, multithreading and performance than webbrowser controls.
TheGateKeeper
Regular Member
- Nov 9, 2011
- 226
- 58
I have learned the hard way that the web browser control is utter sh**. HttpWebRequests are the way to go!
smack
Junior Member
- Feb 1, 2010
- 182
- 80
WebBrowser/HttpWebRequest/Sockets namespace are kind of the Bad/Better/Best options for working with websites in .NET (excluding external libraries such as something like ChillKat).
each of them has certain advantages and disadvantages.
in my opinion the web browser control has the most disadvantages to it. generally speaking it allows for the most rapid development and will handle scripts automatically, however it can't really be multi-threaded in any meaningful way, it is slow and error prone, and as far as i know can only accept one proxy at a time.
HttpWebRequest is considerably better than the web browser control. it can be multi-threaded, it can use multiple proxies, it is more stable, and still provides for fairly rapid development. downsides are that it is still fairly buggy and error prone, especially when it comes to cookie handling, and it does not handle client side script.
System.Net.Sockets is the best option. it is fast, stable, and very robust. the best technique for using it is to spend some time up front creating a re-usable HTTP class with it. while this is the slowest of the three options initially, i feel that the time investment in the beginning pays off quite well later on when you have a pre built HTTP class that can be customized to suite your use cases and provide functionality that you do not get with HttpWebRequest. unfortunately this option does not automatically handle client side script either, and the initial time investment can be more than other options, however these drawbacks are generally offset by the advantages.
in regards to client side script there are various ways to go about getting around it. one of the most basic is to attempt to just turn it off entirely and see if you can still submit your requests successfully. sometimes this works, sometimes it doesn't.
another technique is to reproduce the results of the client script in your .NET code. this can work extremely well if you can track down the specific scripts you need, however it can become very difficult with many sites out there using compressed/minified script files to help decrease page loading times.
one other technique i am aware of, but have not tried myself, is to use a JavaScript interpreter to execute the client script from your managed code. there is a great amount of information about this on Google and even some projects like Jint that are supposed to be designed for just this use case. see the following link: hxxp://jint.codeplex.com/
some of the best tools for working with and debugging client script are the Developer Tools option under the tools menu in Internet Explorer and plugins like Firebug for Firefox. for general HTTP request diagnostics i usually stick to Commview for standard HTTP because it is a fairly light packet logger, and HTTPAnalyzer for requests over HTTPS (SSL). i've also on occasion used WireShark, but personally don't like the UI for it all that much. bonus about WireShark though, it does other protocols like UDP as well.
one final point to consider, sometimes you can get away with using the mobile site, or turning the client script off completely with something like the Firefox plugin NoScript. it doesn't work all the time, but it can often be used to circumvent problem areas like account registration or login.
each of them has certain advantages and disadvantages.
in my opinion the web browser control has the most disadvantages to it. generally speaking it allows for the most rapid development and will handle scripts automatically, however it can't really be multi-threaded in any meaningful way, it is slow and error prone, and as far as i know can only accept one proxy at a time.
HttpWebRequest is considerably better than the web browser control. it can be multi-threaded, it can use multiple proxies, it is more stable, and still provides for fairly rapid development. downsides are that it is still fairly buggy and error prone, especially when it comes to cookie handling, and it does not handle client side script.
System.Net.Sockets is the best option. it is fast, stable, and very robust. the best technique for using it is to spend some time up front creating a re-usable HTTP class with it. while this is the slowest of the three options initially, i feel that the time investment in the beginning pays off quite well later on when you have a pre built HTTP class that can be customized to suite your use cases and provide functionality that you do not get with HttpWebRequest. unfortunately this option does not automatically handle client side script either, and the initial time investment can be more than other options, however these drawbacks are generally offset by the advantages.
in regards to client side script there are various ways to go about getting around it. one of the most basic is to attempt to just turn it off entirely and see if you can still submit your requests successfully. sometimes this works, sometimes it doesn't.
another technique is to reproduce the results of the client script in your .NET code. this can work extremely well if you can track down the specific scripts you need, however it can become very difficult with many sites out there using compressed/minified script files to help decrease page loading times.
one other technique i am aware of, but have not tried myself, is to use a JavaScript interpreter to execute the client script from your managed code. there is a great amount of information about this on Google and even some projects like Jint that are supposed to be designed for just this use case. see the following link: hxxp://jint.codeplex.com/
some of the best tools for working with and debugging client script are the Developer Tools option under the tools menu in Internet Explorer and plugins like Firebug for Firefox. for general HTTP request diagnostics i usually stick to Commview for standard HTTP because it is a fairly light packet logger, and HTTPAnalyzer for requests over HTTPS (SSL). i've also on occasion used WireShark, but personally don't like the UI for it all that much. bonus about WireShark though, it does other protocols like UDP as well.
one final point to consider, sometimes you can get away with using the mobile site, or turning the client script off completely with something like the Firefox plugin NoScript. it doesn't work all the time, but it can often be used to circumvent problem areas like account registration or login.
smack
Junior Member
- Feb 1, 2010
- 182
- 80
one last thing i forgot to mention that should make the future interesting is what Microsoft is doing with their platform.
it's my understanding that with the launch of the newest iteration of Visual Studio and Windows 8 some changes are going to be coming. the shift towards the metro app style of development with WPF/SilverLight is going to see JavaScript elevated to the level of a first class citizen. JS is supposed to be fully supported for Windows Metro Apps native to the OS and it will have support in Visual Studio similar to how C#/VB.NET do now with Intellisense and better code validation. so JS will no longer be relegated to something that is only used for client side execution on websites.
C#/VB.NET are getting promoted to have a higher degree of access to OS resources similar to how lower level languages like C++ have today. allegedly you're going to be able to work directly with OS level functions without the need to get handles or through the use of IntPtr's.
it will be interesting to see how this all plays out, but from the new marketing materials it sounds like there may be a day in the fairly near future where you could theoretically just snag the client script files from a web site and place them directly in to your .NET project for seamless use with your other managed code modules.
it's my understanding that with the launch of the newest iteration of Visual Studio and Windows 8 some changes are going to be coming. the shift towards the metro app style of development with WPF/SilverLight is going to see JavaScript elevated to the level of a first class citizen. JS is supposed to be fully supported for Windows Metro Apps native to the OS and it will have support in Visual Studio similar to how C#/VB.NET do now with Intellisense and better code validation. so JS will no longer be relegated to something that is only used for client side execution on websites.
C#/VB.NET are getting promoted to have a higher degree of access to OS resources similar to how lower level languages like C++ have today. allegedly you're going to be able to work directly with OS level functions without the need to get handles or through the use of IntPtr's.
it will be interesting to see how this all plays out, but from the new marketing materials it sounds like there may be a day in the fairly near future where you could theoretically just snag the client script files from a web site and place them directly in to your .NET project for seamless use with your other managed code modules.