1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrapping YellowPages emails

Discussion in 'General Programming Chat' started by Saulyx, Jul 28, 2011.

  1. Saulyx

    Saulyx Junior Member

    Joined:
    Jan 10, 2010
    Messages:
    107
    Likes Received:
    5
    Hey everyone,

    Been looking at scraping YP with PHP, however seeing as CURL does not execute JS, so the email cant be loaded into it's place, anyone have any ideas how to find it?


    Any help appreciated would be great,


    Thank you,
    S
     
  2. hoodkist

    hoodkist Regular Member

    Joined:
    Jun 20, 2011
    Messages:
    241
    Likes Received:
    33
    Occupation:
    Web Designer and advertiser in marketing new softw
    Location:
    Beta testing your software
    Yes use an email scraping tool to scrape yellow book
    That's what I use there's a new email scraper out fry I've been using that does a wonderfull job with this.
     
  3. kingscript

    kingscript Registered Member

    Joined:
    Jun 3, 2009
    Messages:
    64
    Likes Received:
    9
    Occupation:
    No the correct term is Ok u pay son!
    Location:
    Online
    Whats the name of the software? I dont mind purchasing it.
     
  4. Saulyx

    Saulyx Junior Member

    Joined:
    Jan 10, 2010
    Messages:
    107
    Likes Received:
    5
    Thing is, I need to use php to scrape them, so I cant use software... any other ideas?
     
  5. chatmasta

    chatmasta Junior Member

    Joined:
    Sep 1, 2007
    Messages:
    122
    Likes Received:
    38
    Curl does not execute Javascript, but it CAN replicate what the Javascript is doing. To see exactly what that is, download a FF plugin called "Live HTTP Headers" (google it, I can't link). Go to a site, open Live HTTP Headers, and perform your actions. It will show the requests exactly as they go through. Chances are, the javascript is sending POST requests to get data that you need. Just replicate those requests with curl and you will be all good. :D