1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

problem with extracting form hidden fields value

Discussion in 'PHP & Perl' started by maximviper, Feb 21, 2011.

  1. maximviper

    maximviper BANNED BANNED

    Joined:
    Oct 25, 2010
    Messages:
    338
    Likes Received:
    86
    Hello .

    i was wondering if anyone could guide me a bit about how to extract hidden form fields while login or form posting.
    he problem is ,as those fields are dynamic so i have to extract them on fly before setting CURLOPT_POSTFIELDS .

    i tried to save the whole html page in a text file with he following :

    $fp = fopen('form.html', 'w');
    curl_setopt($ch, CURLOPT_FILE, $fp);

    but that doesnt help. bcos i m not able to reopen the file. i ave no idea.

    PHP:
    function curl_post($site,$postdata){


        
    $ch curl_init();
        
    curl_setopt($chCURLOPT_RETURNTRANSFERTRUE);
        
    curl_setopt($chCURLOPT_COOKIEFILE"cookie.txt");
         
    curl_setopt($chCURLOPT_USERAGENT"Mozilla/5.0 (Windows NT 5.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11");
        
    curl_setopt($chCURLOPT_TIMEOUT40);
        
    curl_setopt($chCURLOPT_URL$site);
        
    curl_setopt($chCURLOPT_HEADERTRUE);
        
    //curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);    // excluded bcos it throws error about safe mode
        
    curl_setopt($chCURLOPT_USERAGENT$_SERVER['HTTP_USER_AGENT']);
        
    curl_setopt($chCURLOPT_POSTTRUE);
        
        
    //saving html page to file on server . this part works
        
    $fp fopen('inbox.html''w');
        
    curl_setopt($chCURLOPT_FILE$fp);
        
    fclose($fp);
        
        
    //opening the file . not works
        
    $fh fopen('inbox.html''r');
        if(
    $fh)
        {
            echo 
    "file not opened";
        }
        else
        {
            echo 
    "file opened";
        }
        echo 
    "filesize = ".filesize($myFile);
        
    $theData fread($fhfilesize($myFile));
        
    fclose($fh);
        echo 
    $theData;
        
        
    //for extracting toekn .no use for now
        // $regex = '/token="(.+?)">/';
        // preg_match($regex,$data,$match);
        // $str=$match[1];
        // echo $str;    
        
        //curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);   //postdata will be modified according to token extracting. no need unless i extract token
       
       
    $response curl_exec ($ch); // execute the curl command

    }
    Is it possible to store $ch data into a buffer variable so that i wont have to save the whole webpage and reopen again ?

    postdata is inform of :
    Code:
    title=$title&FCKeditor1=$article&access=private&vtok=68d6e4f4be9eab2409cb57469370950d&submitted=Posting+Blog...Please+Wait...
    Also CURLOPT_FOLLOWLOCATION always trows error bcos of something related to safe mode .i coudnt figured out a solution yet
     
  2. maximviper

    maximviper BANNED BANNED

    Joined:
    Oct 25, 2010
    Messages:
    338
    Likes Received:
    86
    somebody please help.....
     
  3. Spitman

    Spitman Registered Member

    Joined:
    Mar 17, 2009
    Messages:
    56
    Likes Received:
    20
    I'm confused by what your problem is. In 'intended use' the process would be:

    > User requests page
    > Server sends page to user's browser (including hidden fields)
    > User fills in the form in browser
    > User submits form to server
    > Server processes form data
    > Sever sends response to users browser

    Why not just do this

    > cURL to request page with form on it
    > Use regex / string manipulation to extract hidden fields
    > new cURL to submit form including extracted hidden field values
     
  4. gimme4free

    gimme4free Executive VIP Jr. VIP Premium Member

    Joined:
    Oct 22, 2008
    Messages:
    1,907
    Likes Received:
    1,938
    Home Page:
    I'm lost as to what you are trying to do, are you trying to POST data & retrieve content in the same call?

    The normal procedure would be:
    1) Go To Login Page > Scrape Hidden Form Fields.
    2) Reform Scraped Data Within PHP & POST To Login Form
     
  5. maximviper

    maximviper BANNED BANNED

    Joined:
    Oct 25, 2010
    Messages:
    338
    Likes Received:
    86
    login process is not a problem as i m albe to login easily but the problem is making a post (like article posting)
    as the hidden token value is dynamic and it changes at every call therefore if i first extract the value in one call then post the data in another call then in that again the token value is already been changed so i cant use the old token value. thats the problem here.
    i need to find a way to get the token and post the article in the same call.
     
  6. gimme4free

    gimme4free Executive VIP Jr. VIP Premium Member

    Joined:
    Oct 22, 2008
    Messages:
    1,907
    Likes Received:
    1,938
    Home Page:
    As per step 1 of my last post:
    1) Go To Login Page > Scrape Hidden Form Fields.

    You must parse the scraped data for dynamic values & then reform the data. I added a function in your other thread that does exactly this:
    PHP:
    function getHiddenFields($html){
        
    $ret = array();
        if (
    preg_match_all('#(<input[^>]*type="hidden"[^>]*>)#is'$html$matches)){
            
    $hfields $matches[1];
            foreach (
    $hfields as $hf) {
                if (
    preg_match('#name="([^"]+)"#is'$hf$matches)){
                    
    $hf_name $matches[1];
                    
    $hf_val '';
                    if (
    preg_match('#value="([^"]+)"#is'$hf$matches)){
                        
    $hf_val $matches[1];
                        }
                    
    $ret[] = urlencode(htmlspecialchars_decode($hf_name)).'='.urlencode(htmlspecialchars_decode($hf_val));
                    }
                }
            }
        return 
    $ret;
        }
     
    • Thanks Thanks x 1
  7. Spitman

    Spitman Registered Member

    Joined:
    Mar 17, 2009
    Messages:
    56
    Likes Received:
    20
    Can you tell us what site you are trying to post to?

    This doesn't make sense really because when a normal user visits the page with the form on the hidden fields are downloaded to their browser, they do no change between then and the time they press submit to send the form back to the server.

    I suspect your problem doesn't actually involve the hidden fields but more likely cookies or session data.
     
  8. maximviper

    maximviper BANNED BANNED

    Joined:
    Oct 25, 2010
    Messages:
    338
    Likes Received:
    86
    @gimmeforme

    Thanks for replying. but the wode u posted is not working in this case bcos the
    the $html value that is returned in nothing but "1"

    PHP:
    function curl_grab_page($site){

        
    $ch curl_init();
        
    curl_setopt($chCURLOPT_RETURNTRANSFER0);
         
    curl_setopt($chCURLOPT_USERAGENT"Mozilla/5.0 (Windows NT 5.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11");
        
    curl_setopt($chCURLOPT_COOKIEFILE"cookie.txt");
        
    curl_setopt($chCURLOPT_URL$site);
        
    $fp fopen('inbox.html''w');
        
    curl_setopt($chCURLOPT_FILE$fp);
    //    curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);

        
        
    $html=curl_exec($ch);

        return 
    $html// execute the curl command
            
        
    fclose($fp);
        
        
    }
     
    $data curl_grab_page($site)
    echo 
    $data;
    i guess its bcos i cant use CURLOPT_FOLLOWLOCATION as it always throws error bcos of something related to safe mode .i coudnt figured out a solution yet

    however i really dont understand how the correct html page is getting saved into the inbox.html file .still if i return the value of $html , all it echos is value "1"

    if you are willing to help then i can tell the actual website i m working on witht the whole code. .
     
  9. gimme4free

    gimme4free Executive VIP Jr. VIP Premium Member

    Joined:
    Oct 22, 2008
    Messages:
    1,907
    Likes Received:
    1,938
    Home Page:
    It is because you have disabled returntransfer, you need to re-enable it:
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
     
  10. gimme4free

    gimme4free Executive VIP Jr. VIP Premium Member

    Joined:
    Oct 22, 2008
    Messages:
    1,907
    Likes Received:
    1,938
    Home Page:
    Also, CURLOPT_FILE will override returntransfer as it is putting the contents of the URL into the chosen file but it will not actually transfer the HTML of the page, it is returntransfer that needs to be enabled if you want to use the data within your script.
     
  11. maximviper

    maximviper BANNED BANNED

    Joined:
    Oct 25, 2010
    Messages:
    338
    Likes Received:
    86
    sir actaully i was testin with return transfer to '1' but it was not working. before i posted the code here i was playing with the code a bit and forgot to change it to "1" .
    i m PMing you the whole code with all details. please check if you have time
     
  12. maximviper

    maximviper BANNED BANNED

    Joined:
    Oct 25, 2010
    Messages:
    338
    Likes Received:
    86
    thanks to gimme4free i finally able to make it work.
    now i m working on another site but this one is more complicated. it is doing md5 encryption on the fly with

    var res = MD5(chal + MD5(pass));

    i wonder how can i mimic it in php. i already hav value of 'chal' and 'pass' but to get value of 'res' i will have to do this md5 stuff.
    i tried a few thing but the md5 value i m getting is different :(

    i tried this :


    PHP:
    //$res=MD5($chal.MD5($pass));
    and
    PHP:
    $temp1=md5($pass);
    echo 
    "<BR> md5(pass) = $temp1";

    $temp2=$chal $temp1;
    echo 
    "<BR> chal + temp1 = $temp2";

    $temp3=md5($temp2);
    echo 
    "<BR> $temp3";
    but i guess i m doing it worng.
    can anyone help me with this ?
     
  13. Crazy

    Crazy Jr. Executive VIP

    Joined:
    Jun 13, 2009
    Messages:
    640
    Likes Received:
    319
    Occupation:
    VB, C#, XHTML, CSS, PHP, MySQL, JavaScript, jQuery
    Location:
    Everywhere
    Have you tried md5($chal . md5($pass))

    Edit: fixed that.
     
    • Thanks Thanks x 2
  14. maximviper

    maximviper BANNED BANNED

    Joined:
    Oct 25, 2010
    Messages:
    338
    Likes Received:
    86
    thanks for replying but isnt it same as
    $res=MD5($chal.MD5($pass));

    ??? or spaces matters?
     
  15. Crazy

    Crazy Jr. Executive VIP

    Joined:
    Jun 13, 2009
    Messages:
    640
    Likes Received:
    319
    Occupation:
    VB, C#, XHTML, CSS, PHP, MySQL, JavaScript, jQuery
    Location:
    Everywhere
    Oh sorry I missed that. No that's the same the space between concatenation command doesn't matter. Are you sure your chal & pass values are the same values being handled by the script on the site? Because if they are that should work.
     
  16. maximviper

    maximviper BANNED BANNED

    Joined:
    Oct 25, 2010
    Messages:
    338
    Likes Received:
    86
    i guess i have the values of chal and pass . as

    the form is having this code snippet

    <form method='post' action='update.bml' id='updateForm' name='updateForm'>

    Code:
    <input type='hidden' name="lj_form_auth" value="c0:1298160000:3264:86400:bQKpqrrUSo-34919554-15:5dfc45dacbdf95ee4321f4814ff65443" /><div class='appwidget appwidget-entryform' id='LJWidget_7'>
    <div id='entry-form-wrapper'><input type='hidden' name='chal' id='login_chal' value='c0:1298160000:3264:3600:eSZQUdnXEHx3gTr9Yfmn:b1e1f87c17948e337f0da562fd8bd72d' /><input type='hidden' name='response' id='login_response' value='' />
    and pass is ,i guess the account password.