Scrape page and find if a followed link exists
- Thread starter Runatic
- Start date
hustleharderrrr
Junior Member
- Jan 2, 2012
- 109
- 33
I would get the page using cUrl, then parsing the result dom for the anchor with preg_match and your url, then check if the anchor has a nofollow tag with preg_match
This library makes dom parsing pretty easy. It has similarities with jquery if you are familiar with it...
http : // simplehtmldom . sourceforge . net
pseudo example:
http : // simplehtmldom . sourceforge . net
pseudo example:
Code:
foreach($html->find('a') as $element) {
echo $element->href . '<br>';
//do some pregmatch
}
Last edited:
An alternative would be to parse the content with jquery e.g. like this:
You could e.g. store all found links via ajax request to another script that stores the stuff in a database. And then after finishing get an unscraped url from the database via ajax and call the above script -> name . php ? url=...Little bot like...
P.S.: echo file_get_contents( isset($_REQUEST['url']) ? $_REQUEST['url'] : 'start url' ) and after that include jquery and parse with it
Code:
grrr tried to post an example code here but the moderation system keeps blocking me :-/You could e.g. store all found links via ajax request to another script that stores the stuff in a database. And then after finishing get an unscraped url from the database via ajax and call the above script -> name . php ? url=...Little bot like...
P.S.: echo file_get_contents( isset($_REQUEST['url']) ? $_REQUEST['url'] : 'start url' ) and after that include jquery and parse with it
Last edited: