problem with extracting form hidden fields value

maximviper · Feb 21, 2011

Hello .

i was wondering if anyone could guide me a bit about how to extract hidden form fields while login or form posting.
he problem is ,as those fields are dynamic so i have to extract them on fly before setting CURLOPT_POSTFIELDS .

i tried to save the whole html page in a text file with he following :

$fp = fopen('form.html', 'w');
curl_setopt($ch, CURLOPT_FILE, $fp);

but that doesnt help. bcos i m not able to reopen the file. i ave no idea.

PHP:

function curl_post($site,$postdata){


    $ch = curl_init();
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
    curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
     curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 5.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11");
    curl_setopt($ch, CURLOPT_TIMEOUT, 40);
    curl_setopt($ch, CURLOPT_URL, $site);
    curl_setopt($ch, CURLOPT_HEADER, TRUE);
    //curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);    // excluded bcos it throws error about safe mode
    curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
    curl_setopt($ch, CURLOPT_POST, TRUE);
    
    //saving html page to file on server . this part works
    $fp = fopen('inbox.html', 'w');
    curl_setopt($ch, CURLOPT_FILE, $fp);
    fclose($fp);
    
    //opening the file . not works
    $fh = fopen('inbox.html', 'r');
    if($fh)
    {
        echo "file not opened";
    }
    else
    {
        echo "file opened";
    }
    echo "filesize = ".filesize($myFile);
    $theData = fread($fh, filesize($myFile));
    fclose($fh);
    echo $theData;
    
    //for extracting toekn .no use for now
    // $regex = '/token="(.+?)">/';
    // preg_match($regex,$data,$match);
    // $str=$match[1];
    // echo $str;    
    
    //curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);   //postdata will be modified according to token extracting. no need unless i extract token
   
   $response = curl_exec ($ch); // execute the curl command

}

Is it possible to store $ch data into a buffer variable so that i wont have to save the whole webpage and reopen again ?

postdata is inform of :

Code:

title=$title&FCKeditor1=$article&access=private&vtok=68d6e4f4be9eab2409cb57469370950d&submitted=Posting+Blog...Please+Wait...

Also CURLOPT_FOLLOWLOCATION always trows error bcos of something related to safe mode .i coudnt figured out a solution yet

maximviper · Feb 21, 2011

somebody please help.....

Spitman · Feb 21, 2011

I'm confused by what your problem is. In 'intended use' the process would be:

> User requests page
> Server sends page to user's browser (including hidden fields)
> User fills in the form in browser
> User submits form to server
> Server processes form data
> Sever sends response to users browser

Why not just do this

> cURL to request page with form on it
> Use regex / string manipulation to extract hidden fields
> new cURL to submit form including extracted hidden field values

gimme4free · Feb 21, 2011

I'm lost as to what you are trying to do, are you trying to POST data & retrieve content in the same call?

The normal procedure would be:
1) Go To Login Page > Scrape Hidden Form Fields.
2) Reform Scraped Data Within PHP & POST To Login Form

maximviper · Feb 21, 2011

login process is not a problem as i m albe to login easily but the problem is making a post (like article posting)
as the hidden token value is dynamic and it changes at every call therefore if i first extract the value in one call then post the data in another call then in that again the token value is already been changed so i cant use the old token value. thats the problem here.
i need to find a way to get the token and post the article in the same call.

gimme4free · Feb 21, 2011

sourabhswarnkar said:
login process is not a problem as i m albe to login easily but the problem is making a post (like article posting)
as the hidden token value is dynamic and it changes at every call therefore if i first extract the value in one call then post the data in another call then in that again the token value is already been changed so i cant use the old token value. thats the problem here.
i need to find a way to get the token and post the article in the same call.

As per step 1 of my last post:
1) Go To Login Page > Scrape Hidden Form Fields.

You must parse the scraped data for dynamic values & then reform the data. I added a function in your other thread that does exactly this:

PHP:

function getHiddenFields($html){
    $ret = array();
    if (preg_match_all('#(<input[^>]*type="hidden"[^>]*>)#is', $html, $matches)){
        $hfields = $matches[1];
        foreach ($hfields as $hf) {
            if (preg_match('#name="([^"]+)"#is', $hf, $matches)){
                $hf_name = $matches[1];
                $hf_val = '';
                if (preg_match('#value="([^"]+)"#is', $hf, $matches)){
                    $hf_val = $matches[1];
                    }
                $ret[] = urlencode(htmlspecialchars_decode($hf_name)).'='.urlencode(htmlspecialchars_decode($hf_val));
                }
            }
        }
    return $ret;
    }

Spitman · Feb 21, 2011

sourabhswarnkar said:
login process is not a problem as i m albe to login easily but the problem is making a post (like article posting)
as the hidden token value is dynamic and it changes at every call therefore if i first extract the value in one call then post the data in another call then in that again the token value is already been changed so i cant use the old token value. thats the problem here.
i need to find a way to get the token and post the article in the same call.

Can you tell us what site you are trying to post to?

This doesn't make sense really because when a normal user visits the page with the form on the hidden fields are downloaded to their browser, they do no change between then and the time they press submit to send the form back to the server.

I suspect your problem doesn't actually involve the hidden fields but more likely cookies or session data.

maximviper · Feb 21, 2011

@gimmeforme

Thanks for replying. but the wode u posted is not working in this case bcos the
the $html value that is returned in nothing but "1"

PHP:

function curl_grab_page($site){

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
     curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 5.1; rv:2.0b11) Gecko/20100101 Firefox/4.0b11");
    curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
    curl_setopt($ch, CURLOPT_URL, $site);
    $fp = fopen('inbox.html', 'w');
    curl_setopt($ch, CURLOPT_FILE, $fp);
//    curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);

    
    $html=curl_exec($ch);

    return $html; // execute the curl command
        
    fclose($fp);
    
    
}
 $data = curl_grab_page($site)
echo $data;

i guess its bcos i cant use CURLOPT_FOLLOWLOCATION as it always throws error bcos of something related to safe mode .i coudnt figured out a solution yet

however i really dont understand how the correct html page is getting saved into the inbox.html file .still if i return the value of $html , all it echos is value "1"

if you are willing to help then i can tell the actual website i m working on witht the whole code. .

gimme4free · Feb 21, 2011

It is because you have disabled returntransfer, you need to re-enable it:
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

gimme4free · Feb 21, 2011

Also, CURLOPT_FILE will override returntransfer as it is putting the contents of the URL into the chosen file but it will not actually transfer the HTML of the page, it is returntransfer that needs to be enabled if you want to use the data within your script.

maximviper · Feb 21, 2011

sir actaully i was testin with return transfer to '1' but it was not working. before i posted the code here i was playing with the code a bit and forgot to change it to "1" .
i m PMing you the whole code with all details. please check if you have time

maximviper · Feb 21, 2011

thanks to gimme4free i finally able to make it work.
now i m working on another site but this one is more complicated. it is doing md5 encryption on the fly with

var res = MD5(chal + MD5(pass));

i wonder how can i mimic it in php. i already hav value of 'chal' and 'pass' but to get value of 'res' i will have to do this md5 stuff.
i tried a few thing but the md5 value i m getting is different

i tried this :

PHP:

//$res=MD5($chal.MD5($pass));

and

PHP:

$temp1=md5($pass);
echo "<BR> md5(pass) = $temp1";

$temp2=$chal + $temp1;
echo "<BR> chal + temp1 = $temp2";

$temp3=md5($temp2);
echo "<BR> $temp3";

but i guess i m doing it worng.
can anyone help me with this ?

Crazy · Feb 21, 2011

Have you tried md5($chal . md5($pass))

Edit: fixed that.

maximviper · Feb 21, 2011

Crazy. said:
Have you tried md5($chal . md5($pass))

Edit: fixed that.

thanks for replying but isnt it same as
$res=MD5($chal.MD5($pass));

??? or spaces matters?

Crazy · Feb 21, 2011

Oh sorry I missed that. No that's the same the space between concatenation command doesn't matter. Are you sure your chal & pass values are the same values being handled by the script on the site? Because if they are that should work.

maximviper · Feb 21, 2011

i guess i have the values of chal and pass . as

the form is having this code snippet

<form method='post' action='update.bml' id='updateForm' name='updateForm'>

Code:

<input type='hidden' name="lj_form_auth" value="c0:1298160000:3264:86400:bQKpqrrUSo-34919554-15:5dfc45dacbdf95ee4321f4814ff65443" /><div class='appwidget appwidget-entryform' id='LJWidget_7'>
<div id='entry-form-wrapper'><input type='hidden' name='chal' id='login_chal' value='c0:1298160000:3264:3600:eSZQUdnXEHx3gTr9Yfmn:b1e1f87c17948e337f0da562fd8bd72d' /><input type='hidden' name='response' id='login_response' value='' />

and pass is ,i guess the account password.

problem with extracting form hidden fields value

maximviper

BANNED

maximviper

BANNED

Spitman

Registered Member

gimme4free

Elite Member

maximviper

BANNED

gimme4free

Elite Member

Spitman

Registered Member

maximviper

BANNED

gimme4free

Elite Member

gimme4free

Elite Member

maximviper

BANNED

maximviper

BANNED

Crazy

Power Member

maximviper

BANNED

Crazy

Power Member

maximviper

BANNED

Main Menu

Marketplace

Making Money

BlackHat World