1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Parsing/Scraping profiles from social networks (facebook, twitter etc)

Discussion in 'PHP & Perl' started by MonsterMag, Sep 22, 2015.

  1. MonsterMag

    MonsterMag Power Member

    Joined:
    May 2, 2015
    Messages:
    602
    Likes Received:
    417
    Occupation:
    Self
    Location:
    My Journey Discussion
    I'm not a scripter myself, but I was thinking of opening a WWE (Wrestling) website, where I have a single page for every Wrestler and display their updates/statuses from different social networks like facebook, twitter, instagram etc. I don't want to embed their feed, I want to have everything in my own database, including the pictures hosted on my own server.
    I found this snippet for facebook fanpages:

    Code:
    access_token = 'YOUR_ACCESS_TOKEN'fanpage = 'fanpagename'
    startDate = '09/22/12'
    endDate = '09/22/15'
    url = https://graph.facebook.com/'fanpage'?fields=posts.limit(500).since('startDate').until('endDate').fields(message)&access_token=%s' % (access_token)
    Not 100% sure what it returns (don't know how to make an API call) but as far as I understand, this should give me the last 500 statusupdates of that fanpage which I can then re-create on my website.
    I'm looking for solutions for other social networks like twitter and instagram. All I found for twitter till now seems to be a little old and probably outdated. For instagram I couldn't find anything at all. Other networks I would be interested in: Snapchat, Pinterest, G+, tumblr, vine and youtube.

    This is an early stage project, if I go for it, I need someone to do the scripts for me so if you can help me out here, I might hire you if you want :)

    TL;DR:
    Parse statusupdates from all social networks, curate new on own website.
     
  2. bigideaguy

    bigideaguy Newbie

    Joined:
    May 3, 2010
    Messages:
    23
    Likes Received:
    3
    Occupation:
    Webmaster
    Location:
    Houston, TX
    Home Page:
    What you are looking for is their API. Most have pretty detailed documentation of how to use their SDK but generally you are simply making a post request and getting the json data it returns.

    What you have posted above is the facebook graph api
    Code:
    developers.facebook.com/docs/graph-api/reference/page
    
    You can also get statuses from someones Twitter page like so
    Code:
    dev.twitter.com/rest/reference/get/statuses/user_timeline
    Snapchat:
    Code:
    github.com/JorgenPhi/php-snapchat
    Instagram:
    Code:
    github.com/cosenary/Instagram-PHP-API
    Pinterest:
    Code:
    github.com/dirkgroenen/Pinterest-API-PHP
    G+ (You will also find Youtube in the Developer Console):
    Code:
    developers.google.com/+/web/samples/php
    Tumblr:
    Code:
    github.com/tumblr/tumblr.php
    Vine:
    Code:
    github.com/ptariche/Vine-PHP
    Remember that each API probably has some limitations, either by number of requests per minute or per day or both. Some APIs are better and friendlier than other.
     
    • Thanks Thanks x 1
  3. mantic0re

    mantic0re Junior Member

    Joined:
    May 29, 2015
    Messages:
    118
    Likes Received:
    9
    If they don't have an API you can just do HTTP requests and use an existing library like guzzle to parse/extract the data that you need. Alternatively you can just do CURL requests yourself without any additional libraries and parse the data using the built in PHP library(regexes/explodes etc). The latter is not recommended. At any rate you will need to do some coding.