1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Tool to scrape specific data from 1 site?

Discussion in 'Blogging' started by vwvr76, Apr 18, 2010.

  1. vwvr76

    vwvr76 Newbie

    Joined:
    Oct 25, 2009
    Messages:
    20
    Likes Received:
    0
    I'm looking to start up an auto-blog on MMA/Boxing/etc, I'd like to use Wordpress but it's not a must have.

    Ideally I'd like to set it up so a user can click the link on any of the events I'll have listed, and you'll see info like
    Date
    Fightcard
    Description

    Is there something I could use that will
    1. Scrape this data from a site I specify
    2. Enter the data on my site

    and is there a way to have it display according to my specs? Like

    EVENT TITLE
    Date in MM/DD/YR

    <img>

    Fighter A vs B
    Fighter C vs D
    etc. etc.

    Show description, recap.

    I realize there may not be an out of box solution to this, so any tips on even where I should begin learning would be great.
     
  2. Grizzy

    Grizzy Senior Member

    Joined:
    Nov 11, 2008
    Messages:
    919
    Likes Received:
    999
    Are you familiar with PHP (cURL in particular)? If you are, this can be done with relative ease. I do this kind of stuff with wordpress all the time.
     
  3. vwvr76

    vwvr76 Newbie

    Joined:
    Oct 25, 2009
    Messages:
    20
    Likes Received:
    0
    No unfortunately I'm not that familiar with it. Besides learning the basics on php, what sort of scripts or functions should I be looking into?
     
  4. Grizzy

    Grizzy Senior Member

    Joined:
    Nov 11, 2008
    Messages:
    919
    Likes Received:
    999
    For php you basically need to know how to use lib cURL and the various functions that can be used to parse data. In addition to a little bit of wordpress api knowledge, you need to be familiar with the wp_insert_post() function and maybe the update_post_meta() function.

    cURL to scrape the page
    php to parse the required data out
    wordpress to insert the data into your blog and structure it the way you need.
     
    • Thanks Thanks x 1
  5. vwvr76

    vwvr76 Newbie

    Joined:
    Oct 25, 2009
    Messages:
    20
    Likes Received:
    0
    Wow, I have a lot of reading to do. Hopefully I don't need to be an expert to do this sort of thing. If so I may try to outsource this, thanks man.