[ASK] Selective Scraping

T

twisted_one

Power Member

Oct 11, 2009

521

97

• Mar 10, 2011

• #1

There is a basic HTML website static website. I need only selective parts of that site. The site somewhat looks like what I have attached in the Basic HTML.JPG file

That HTML page is fully made of tables, p tags, h1 tags and nothing more. NO DIVS, NO CSS, NO Dynamic content.

How can I grab selective parts of that page. I know in php file_get_contents($url) gets you the whole page but I want only selective.

Lets say for example I want the date and that name Rachel Mathews alone. How can it be done? Any tools or any scripting languages that will help me?

 

Attachments

• 
  Basic HTML Page.jpg

  17.9 KB · Views: 101

madoctopus

Supreme Member

Apr 4, 2010

1,388

3,709

• Mar 10, 2011

• #2

Use simple_html_dom or phpQuery or regex. Whichever you want, any one works.

 

T

twisted_one

Power Member

Oct 11, 2009

521

97

• Mar 10, 2011
• Thread Starter Thread Starter

• #3

MadOctopus,

Thanks for the advice. I am now google'ing those. I am not that good in RegEx so I will try to look at those two options first. In the meantime, do you mind giving me an example?

EDIT:

Okay got even the examples at http://simplehtmldom.sourceforge.net/manual.htm

I now have another query -

Once I have that value in a variable using PHP. is there any way that I can write it in a Notepad (.txt) file?

 

Last edited: Mar 10, 2011

A

abantu

Newbie

Oct 4, 2008

11

22

• Mar 10, 2011

• #4

PHP:

$fp = fopen('data.txt', 'w');
fwrite($fp, '1');
fwrite($fp, '23');
fclose($fp);

 

T

twisted_one

Power Member

Oct 11, 2009

521

97

• Mar 11, 2011
• Thread Starter Thread Starter

• #5

I have a doubt

Code:

<?php
include('simplehtmldom/simple_html_dom.php');
$html = file_get_html('http://somedomain.com?QryParm=1');
echo $html
?>

Let us say for example I will be iterating this code block for QryParm 1 to 10000. I can do that using a for loop. BUT, my problem is this link

Code:

http://somedomain.com?QryParm=1

is redirected to some other URL. How do I handle this URL redirect?

 

Z

zelma143

Power Member

Jun 25, 2010

573

37

• Mar 11, 2011

• #6

try to do with preg_match...

it can help you out...

 

T

twisted_one

Power Member

Oct 11, 2009

521

97

• Mar 14, 2011
• Thread Starter Thread Starter

• #7

Friends,

I managed to find a solution using simple html dom finally Thanks to all those who replied and helped. I did not know about that html dom before until I heard it here. I really thank you guys!

I however have a doubt. As of now,

$html = file_get_html('URL1');

BUT this URL 1 redirects to some other URL 2 on that same domain with some query parameters. How can this be handled in the simple html dom? Any advise please?

 

E

easyroms

Newbie

Nov 5, 2009

15

1

• Mar 14, 2011

• #8

Your best bet is to use CURL,

php.net/manual/en/curl.examples-basic.php

CURL can also follow header redirects.