1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Can someone help me with this script?

Discussion in 'PHP & Perl' started by istuff, Jun 8, 2009.

  1. istuff

    istuff Junior Member

    Mar 2, 2009
    Likes Received:
    Cookie Stuffing
    I'm trying to build a scraper for CL, I have got the majority of it down, but i need help. The script is made up of two components.

    first component gets the links for all the postings. second components finds emails in the page in which you provide the url. Can someone tell me how to intergrate the $url (which spews out the links) into the scraper which gets the url from a form. I tried outputting the $url into the form and clicking submit however that doesnt output all the links into the form. I'd appreciate it if someone can help.
  2. drdankmendez

    drdankmendez Junior Member

    May 30, 2008
    Likes Received:
    In front of my computer
    Why not have the first part collect the links and add them into a file or db table, then have the second component just loop through the list?

    If you are asking for specific instructions on integrating two scripts, it is hard for me to explain without seeing the scripts, but from what I gather the second script is executed by way of form post so you would need to find the section of the code that is executed when the form is submitted. From there you could create a function from that code then run a loop through all your urls calling the function on each url.

    Hope this helps.
  3. zappak

    zappak Junior Member

    Mar 8, 2007
    Likes Received:
    I'll tell you what I did.

    First I saves all countries of USA.
    then saved all categories.
    After then, got each country and category in loop and fetched all links one by one. It took 5 hours but fetched over 300k listings. and stored all in notepad.

    Finally grabbed each listing and fetch emails from it. so leeched whole CL US section :D
  4. 00CivicEX

    00CivicEX Jr. VIP Jr. VIP Premium Member

    Mar 3, 2009
    Likes Received:
    Why not just just cURL and save the data it retreives to a variable...then use a regular expression to search the data for emails and output them into another variable or print them.