1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[WebRequest] Load Html and send a request without reloading

Discussion in 'C, C++, C#' started by jesusidontknow, Aug 24, 2015.

  1. jesusidontknow

    jesusidontknow Newbie

    Joined:
    Aug 24, 2015
    Messages:
    3
    Likes Received:
    0
    Hey !
    It is possible to load the html and send a request without reloading the page to get the values on the page?



    HttpWebRequest requestdata = (HttpWebRequest)WebRequest.Create("myurl");
    HttpWebResponse responsedata = (HttpWebResponse)requestdata.GetResponse();
    StreamReader srdata = new StreamReader(responsedata.GetResponseStream());

    string codedata = srdata.ReadToEnd(); //recup le code source

    string[] data = linkSession(codedata); //recup les informations pour la requete

    string postData = "ma requête ici";

    byte[] arrayBite = Encoding.ASCII.GetBytes(postData);

    HttpWebRequest request = (HttpWebRequest)WebRequest.Create("my url");
    request.Method = WebRequestMethods.Http.Post;
    request.UserAgent = "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
    request.ContentType = "application/x-www-form-urlencoded";
    request.ContentLength = arrayBite.Length;

    Stream dataStream = request.GetRequestStream();
    dataStream.Write(arrayBite, 0, arrayBite.Length);
    dataStream.Close();

    HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    StreamReader sr = new StreamReader(response.GetResponseStream());


    string code = sr.ReadToEnd();
    Console.WriteLine(code);

    But here, when i try to send the request, it reload the page and my request return False. What can i do?
     
  2. 9to5destroyer

    9to5destroyer Jr. VIP Jr. VIP

    Joined:
    Nov 14, 2011
    Messages:
    360
    Likes Received:
    207
    do you mean after you visit the url a second time is refreshed back to the start page. If so you need to handle cookies
     
  3. rootjazz

    rootjazz Jr. VIP Jr. VIP

    Joined:
    Dec 21, 2012
    Messages:
    685
    Likes Received:
    329
    Occupation:
    Developer
    Location:
    UK
    Home Page:
    as above you need to handle cookies

    Also you are not closing your streams (memory leak)


    You want to create yourself some classes to wrap that functionality for you.


    doesn't look like you are POSTing a form either so that isn't needed


    So you can work like

    Code:
    var bot = new Bot(); //creates the cookieContainer and stores it for this "bot" object
    var src= bot.GetPage(url);
    var src2 = bot.GetPage(url);
    ...
    
    you can then expand it out, so you can access HtmlAgility dom parser after your calls and added various helper functions to extract forms, submit forms etc. Then you can make it as realistic as you want (accept gzip, set correct headers etc etc)


    If you haven't already, download fiddler2, learn it, use it. Analyse what is different in your requests compared to your browser. Then understand why and fix your botting classes to make them identical
     
  4. itz_styx

    itz_styx Jr. VIP Jr. VIP

    Joined:
    May 8, 2012
    Messages:
    610
    Likes Received:
    284
    Occupation:
    CEO / Admin / Developer
    Location:
    /dev/mem
    Home Page:
    +1 to use fiddler as rootjazz suggested. he made some good points, but one more thing you might want to consider:
    when using ReadToEnd() you can run into problems sometimes for example when there are connection issues and it would just hang without throwing an exception.
    to get around this, you simply just read a maximum predefined buffer like: char[] buffer = new char[readbuf];
    then just use StreamReader's Read() in a while loop :)