How to scrape Title & Description from Yahoo Answers
- Thread starter issem10
- Start date
You have to make it fully i don't think there is any script for it
let me tell you Process
Input URL by using textbox or string as it in PHP/Perl Category
go to that url using CURL
get response from it
Use regex for title & Description
Simple and also use useragent in Curl method
let me tell you Process
Input URL by using textbox or string as it in PHP/Perl Category
go to that url using CURL
get response from it
Use regex for title & Description
Simple and also use useragent in Curl method
This is exactly how I would do it too. If you are looking to do a mass amount of these, you will more than likely find a pattern or some unique thing once you have done a couple that will allow you to easily progress from one Q/A to another.
I think there is a programming/scripting section here somewhere but here you go.. made it as detailed and simple as I could.
How to use:
Code:
<?
$url=$argv['1'];
// target y answers url
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "$url");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
// page is now in $output variable, lets extract the title
$title_starts_at_position=stripos($output,"<title>") + 7;
$title_ends_at_position=stripos($output,"</title>");
$length_of_the_title=$title_ends_at_position-$title_starts_at_position;
$title_text=substr($output,$title_starts_at_position,$length_of_the_title);
$title_without_yahoo_answers_suffix=str_replace(" - Yahoo! Answers","", $title_text);
echo "\nTitle: $title_without_yahoo_answers_suffix"."\n";
// same with description
$description_starts_at_position=stripos($output,'<div class="content"') + 20;
$remaining_content_from_start_of_description=substr($output,$description_starts_at_position,1000);
$description_ends_at_position=stripos($remaining_content_from_start_of_description,"</div>");
$description=substr($remaining_content_from_start_of_description,1,$description_ends_at_position -1);
echo "Description: ".$description."\n";
?>How to use:
CodingAndStuff
Regular Member
- May 6, 2012
- 239
- 92
I'd use the "Simple HTML Dom Parser" library found here: http://simplehtmldom.sourceforge.net/
Then you'd do something like this:
I didn't test it, but it should work fine.
Then you'd do something like this:
Code:
<?php
require_once("simple_html_dom.php");
$question_id = "Put the question ID here";
$html = file_get_html("http://answers.yahoo.com/question/" . $question_id);
$title = $html->find('h1.subject');
$description = $html->find('div.content');
echo "Title is: " . $title . " and Description is: " . $description;
?>I didn't test it, but it should work fine.