data scrapping problem

Discussion in 'PHP & Perl' started by freotech, Oct 20, 2010.

  1. freotech

    freotech Newbie

    May 7, 2008
    Likes Received:
    I have problem when scrapping data with curl
    once I scrape page using form submit,
    I receive data and everything is ok.
    but when I change in form fields and resubmit, I receive same previous page.
    however I am using new instance for curl.
    I had some useful workaround that
    I wrote two scripts : 1.php that submit form with data1
    and 2.php that submit form with data2.
    and at the end of 1.php I redirect to 2.php
    so once I execute 1.php I got the expected results. from both 1.php and 2.php.

    any idea what is going on??
  2. MaDeuce

    MaDeuce Newbie

    Oct 24, 2008
    Likes Received:
    Austin, TX
    When you say that you are using a new instance for curl, do you mean that you are doing a curl_init() before your second call? If not, that could be your problem.

    There isn't much to go wrong. I'm sure you know the sequence:

    (1) curl_init()

    followed by setting needed options...

    (2) curl_setopt()

    and finally...

    (3) curl_exec()

    Another possibility -- are you setting CURLOPT_URL and CURLOPT_POST correctly in your second call to curl_exec()?

    It's gotta be something simple, but sometimes the obvious ones are the toughest.