1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Xpath for Google SERP

Discussion in 'PHP & Perl' started by livewire, Mar 15, 2010.

  1. livewire

    livewire Newbie

    Joined:
    Dec 2, 2009
    Messages:
    8
    Likes Received:
    0
    Im tring to create a URL Scraper in PHP for Google SERP. Since im using Xpath there seem to be a problem.

    When i try it is evaluated in xpather it shows the result
    Code:
    /html/body//a
    it works pretty good and shows all the links that is available in the site which i dont need.

    but When i try the below code in xpather re result is shown
    Code:
    /html/body/div[@id='cnt']/div[@id='res']/div/ol/li/div//cite
    
    but when i enter the xpath in the php evaluate , nothing is shown..

    Can some one please help me to get the proper xpath for the Google search results ??
     
  2. voyevoda

    voyevoda Regular Member Premium Member

    Joined:
    Mar 21, 2010
    Messages:
    217
    Likes Received:
    97
    Location:
    Eastern Front
    You don't always need to start from the root node of an xml document:

    Code:
    //h3/a
    :)

    Which data from the SERPs are you trying to get? The above will get you the actual URLs of the results.
     
  3. thanhclix

    thanhclix Power Member

    Joined:
    Oct 25, 2008
    Messages:
    646
    Likes Received:
    176
    Install Firebug for Firefox, it has XPath capture function.
     
  4. MirniPobjednik

    MirniPobjednik Registered Member

    Joined:
    Mar 12, 2009
    Messages:
    58
    Likes Received:
    10
    Go as simple as possible, view source and find unique attribute then make this
    //*[@attribute='unique_value_of_tag_you_want'] asteriks means any tag. If that does not hel try changing apostrophes from ' to " and vice versa this can cause problems. Simply experiment with different xml until you hit the right one.