Need an IMDB content scraper

chucklechuck · Dec 17, 2008

Does anybody know where i can find a decent imdb content scraper that would allow to show latest releases etc on my site?

Had a scout around here and google but can only find scrapers for home media/personal use etc. Can't see any feeds from their site either!

Any advice would be appreciated

Cheers

John

valwardon · Dec 17, 2008

good qustion you can turn the site into a feed than scrape it .

There is a site that does that but can not find it right now.

*edit jsut found it

Code:

http://www.extralabs.net/rss-wizard.htm

*

chucklechuck · Dec 17, 2008

Awesome, thanks Val

Feeds are a completely new thing to me and will do some quick learning tonight. Looks like it does what i need though. If this can be used for any website as it looks like it can then the possibilities are endless with this.

Great find

markdigerati · Dec 18, 2008

Dont know if this helps, but I just posted a PHP/cURL scraping post.

Using that you could easily make your own iMDB scraper.

neta1o · Dec 18, 2008

http://www.blackhatworld.com/blackhat-seo/php-perl/38082-custom-scrape-tool-php.html

markdigerati · Dec 18, 2008

neta1o said:
http://www.blackhatworld.com/blackhat-seo/php-perl/38082-custom-scrape-tool-php.html

Oh, right. Didn't know someone had already published one. Apologies, just trying to help.

neta1o · Dec 18, 2008

markdigerati said:
Oh, right. Didn't know someone had already published one. Apologies, just trying to help.

It's not for IMDB but it's custom and takes any input, so you can customize it for IMDB

Let me know if you need help b/c I made it

markdigerati · Dec 18, 2008

Ahh okay.

Yes, the class I provided you can customise it. You just pass a URL, start-tag & end-tag and it does the rest for you.

chucklechuck · Dec 18, 2008

Thanks Neta and Mark,

Noob question for you, and apologise as i've quickly looked over those links, but are these tools able to run automatically when the page is loaded.

Say for instance i'm wanting to scrape the "now showing" page on imdb. Will it be updated?

neta1o · Dec 18, 2008

Alright here is a nice general answer that can be used for any page, but the code I'll be pasting here is customized to match chucklechucks request.

You'll need a php page, any of your existing pages

Put this code on the page where you want the current movies listed

Code:

<?php
$url="http://www.imdb.com/nowplaying/";
$beg="<td class="movie ">";
$end="</a>";

$data = file_get_contents($url);
	$regex = '/'.$beg.'(.+?)'.$end.'/';
	$count=1;
	preg_match_all($regex,$data,$match,PREG_SET_ORDER);
	foreach ($match as $result) {
		$link = $result[1];
		$link=strip_tags($link);
		echo $link . '<br>';
	}
?>

I uploaded this example here http://answerexpert.net/movies.php on my site so you can see the result of how it will work. Any time someone loads that page it will always look to IMDB and scrape the newest listings. 100% dynamic

For other interested in using this you just have to change the first 3 listings
url - the website link you want to scrape
beg - beginning code string
end - ending code string

EDIT: Thanks and Rep are appreciated for this freebie

chucklechuck · Dec 18, 2008

Brilliant, exactly what i was looking for. Will play around with it and see what can be done.

Thanks given - how do you rep?

neta1o · Dec 18, 2008

For whatever reason the forum took parts of the code out with the code tags, download the scraping file directly from this post http://www.blackhatworld.com/blackhat-seo/php-perl/38082-custom-scrape-tool-php.html#post400664

It is attached, all files in tact.

Need an IMDB content scraper

chucklechuck

Registered Member

valwardon

BANNED

chucklechuck

Registered Member

markdigerati

Junior Member

neta1o

Regular Member

markdigerati

Junior Member

neta1o

Regular Member

markdigerati

Junior Member

chucklechuck

Registered Member

neta1o

Regular Member

chucklechuck

Registered Member

neta1o

Regular Member

Main Menu

Marketplace

Making Money

BlackHatWorld

Other