1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Need an IMDB content scraper

Discussion in 'Black Hat SEO Tools' started by chucklechuck, Dec 17, 2008.

  1. chucklechuck

    chucklechuck Registered Member

    Joined:
    Dec 2, 2008
    Messages:
    93
    Likes Received:
    53
    Does anybody know where i can find a decent imdb content scraper that would allow to show latest releases etc on my site?

    Had a scout around here and google but can only find scrapers for home media/personal use etc. Can't see any feeds from their site either!

    Any advice would be appreciated

    Cheers

    John
     
  2. valwardon

    valwardon BANNED BANNED

    Joined:
    Apr 9, 2008
    Messages:
    704
    Likes Received:
    867
    good qustion you can turn the site into a feed than scrape it .



    There is a site that does that but can not find it right now.

    *edit jsut found it
    Code:
    http://www.extralabs.net/rss-wizard.htm
    *
     
    • Thanks Thanks x 1
  3. chucklechuck

    chucklechuck Registered Member

    Joined:
    Dec 2, 2008
    Messages:
    93
    Likes Received:
    53
    Awesome, thanks Val

    Feeds are a completely new thing to me and will do some quick learning tonight. Looks like it does what i need though. If this can be used for any website as it looks like it can then the possibilities are endless with this.

    Great find
     
  4. markdigerati

    markdigerati Junior Member

    Joined:
    Nov 21, 2007
    Messages:
    113
    Likes Received:
    28
    Dont know if this helps, but I just posted a PHP/cURL scraping post.

    Using that you could easily make your own iMDB scraper.
     
    • Thanks Thanks x 1
  5. neta1o

    neta1o Regular Member

    Joined:
    Sep 29, 2008
    Messages:
    388
    Likes Received:
    318
    Home Page:
    • Thanks Thanks x 1
  6. markdigerati

    markdigerati Junior Member

    Joined:
    Nov 21, 2007
    Messages:
    113
    Likes Received:
    28
  7. neta1o

    neta1o Regular Member

    Joined:
    Sep 29, 2008
    Messages:
    388
    Likes Received:
    318
    Home Page:
    It's not for IMDB but it's custom and takes any input, so you can customize it for IMDB :)

    Let me know if you need help b/c I made it
     
  8. markdigerati

    markdigerati Junior Member

    Joined:
    Nov 21, 2007
    Messages:
    113
    Likes Received:
    28
    Ahh okay.

    Yes, the class I provided you can customise it. You just pass a URL, start-tag & end-tag and it does the rest for you.
     
  9. chucklechuck

    chucklechuck Registered Member

    Joined:
    Dec 2, 2008
    Messages:
    93
    Likes Received:
    53
    Thanks Neta and Mark,

    Noob question for you, and apologise as i've quickly looked over those links, but are these tools able to run automatically when the page is loaded.

    Say for instance i'm wanting to scrape the "now showing" page on imdb. Will it be updated?
     
    Last edited: Dec 18, 2008
  10. neta1o

    neta1o Regular Member

    Joined:
    Sep 29, 2008
    Messages:
    388
    Likes Received:
    318
    Home Page:
    Alright here is a nice general answer that can be used for any page, but the code I'll be pasting here is customized to match chucklechucks request.

    You'll need a php page, any of your existing pages

    Put this code on the page where you want the current movies listed

    Code:
    <?php
    $url="http://www.imdb.com/nowplaying/";
    $beg="<td class="movie ">";
    $end="</a>";
    
    $data = file_get_contents($url);
    	$regex = '/'.$beg.'(.+?)'.$end.'/';
    	$count=1;
    	preg_match_all($regex,$data,$match,PREG_SET_ORDER);
    	foreach ($match as $result) {
    		$link = $result[1];
    		$link=strip_tags($link);
    		echo $link . '<br>';
    	}
    ?>
    I uploaded this example here http://answerexpert.net/movies.php on my site so you can see the result of how it will work. Any time someone loads that page it will always look to IMDB and scrape the newest listings. 100% dynamic

    For other interested in using this you just have to change the first 3 listings
    url - the website link you want to scrape
    beg - beginning code string
    end - ending code string

    EDIT: Thanks and Rep are appreciated for this freebie ;)
     
    • Thanks Thanks x 3
  11. chucklechuck

    chucklechuck Registered Member

    Joined:
    Dec 2, 2008
    Messages:
    93
    Likes Received:
    53
    Brilliant, exactly what i was looking for. Will play around with it and see what can be done.

    Thanks given - how do you rep?
     
  12. neta1o

    neta1o Regular Member

    Joined:
    Sep 29, 2008
    Messages:
    388
    Likes Received:
    318
    Home Page: