1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[TUTORIAL] Facebook UID's to Emails (write your own scraper!)

Discussion in 'FaceBook' started by Rockonaut, Nov 14, 2014.

  1. Rockonaut

    Rockonaut Registered Member

    Joined:
    Feb 14, 2014
    Messages:
    76
    Likes Received:
    27
    So you have a lot of UIDs? no problem today we are going to convert them to @facebook.com emails and use them for a custom audience! but we are going to make our own scraper for this!

    Many of you have worked with HTML/CSS, some of you maybe worked with JAVA, C++ /C or any other languages for you this will be easy to pickup, if you don't have any experience it will take around a week to learn this :)

    The final script we create:
    Code:
    
    var request = require('request'),
        cheerio = require('cheerio'),
        fs = require('fs');
    
    
    var facebookurl = "http://graph.facebook.com/";
    var email = "@facebook.com"
    
    
    var finalMail = [];
    
    
    var timersetter =  Math.floor((Math.random() * 3000) + 300);
    
    
    
    
    var userids = fs.readFileSync('uidsfiles.txt').toString().split("\n");
    
    
    
    
      for (i in userids) {
          function processNextUrl(index) {
              var userid = userids[index];
    
    
              totalurl = facebookurl + userid;
              request(totalurl, function(error, response, html) {
                  if (!error && response.statusCode == 200) {
                 
    
    
                      var obj = JSON.parse(html);
    
    
                      finalMail.push(obj.username + email);
    
    
                  
                      
                        console.log(obj);
                        console.log(finalMail.length + " done")
                      console.log(userids.length - finalMail.length + "to do");
    
    
    
    
                     // console.log(finalMail.length);
    
    
                      fs.writeFile("emails.txt", finalMail + "\n", function(err) {
                          if (err) {
                              console.log(err);
                          } else {
    
    
                          }
                      });
    
    
                  }
                  
                  if (userids.length > index - 1) {
                      setTimeout(function() {
                          processNextUrl(index + 1);
                      }, timersetter);
                  }
              });
          }
      }
    
    
    
    
      processNextUrl(0);
    This is javascript, well with some node.js. but what happens?

    Facebook graph gives you user info based on their UID, for example:

    Code:
    http://graph.facebook.com/100001525702202
    output:

    Code:
    {   "id": "100001525702202",
       "first_name": "\u0421\u0430\u043d\u044f",
       "gender": "male",
       "last_name": "\u0421\u0435\u0440\u0433\u0435\u0435\u0432",
       "link": "https://www.facebook.com/stuk.koles",
       "locale": "ru_RU",
       "name": "\u0421\u0430\u043d\u044f \u0421\u0435\u0440\u0433\u0435\u0435\u0432",
       "username": "stuk.koles" }
    So when we add a UID behind graph.facebook.com we get an JSON file with their info! what we are intrested in are the usernames, if you add '@facebook.com' behind them you have their fb email ready to upload to the facebook powereditor.

    but doing this by hand takes alot of time.. what we are going to make is a simple command line tool!

    Screen Shot 2014-11-14 at 8.44.46 AM.png

    First you need to install some software:

    http://www.sublimetext.com/
    or any other text editor (no word etc.)

    http://nodejs.org/

    and
    https://github.com/request/request
    https://github.com/cheeriojs/cheerio

    Code:
    [COLOR=#333333][FONT=Consolas]npm install cheerio
    [/FONT][/COLOR][COLOR=#333333][FONT=Consolas]npm install request[/FONT][/COLOR]
    These final 2 can be installed with the command line, if you are new to the terminal read this book:
    http://cli.learncodethehardway.org/book/ learn code the hardway is a great resource of info, check them out!

    That was our setup, now lets start coding!

    What happens first:
    Code:
    var request = require('request'),    cheerio = require('cheerio'),
        fs = require('fs');
    
    
    var facebookurl = "http://graph.facebook.com/";
    var userdata = [];
    var email = "@facebook.com"
    
    
    var finalMail = [];
    
    
    var timersetter =  Math.floor((Math.random() * 3000) + 300);
    
    
    
    
    var userids = fs.readFileSync('uidsfiles.txt').toString().split("\n");
    Here we call our modules like cheerio, request and declare variables, userids is interesting because we use a file to insert an array with the user ids!


    The rest of the code is explained in the comments // <-- is a comment
    Code:
    
    var request = require('request'),
        cheerio = require('cheerio'),
        fs = require('fs');
    
    
    var facebookurl = "http://graph.facebook.com/";
    var userdata = [];
    var email = "@facebook.com"
    
    
    var finalMail = [];
    
    
    //random time between 300 - 3000 ms
    var timersetter =  Math.floor((Math.random() * 3000) + 300);
    
    
    //grap the UIDS from the file! 
    var userids = fs.readFileSync('uidsfiles.txt').toString().split("\n");
    
    
    
    
      for (i in userids) {
          function processNextUrl(index) {
              var userid = userids[index];
    
    
              //create an new URL with the next UID!
              totalurl = facebookurl + userid;
              request(totalurl, function(error, response, html) {
                  if (!error && response.statusCode == 200) {
                  
                      //take the JSON data and make it an object!
                      var obj = JSON.parse(html);
    
    
                        //push the obj.username data + the @facebook.com to finalMail array! 
                      finalMail.push(obj.username + email);
    
    
                  
                      //print stuff out to the command line! just a reminder for myself to show whats happening! 
                        console.log(obj);
                        console.log(finalMail.length + " done")
                      console.log(userids.length - finalMail.length + "to do");
    
    
    
    
                      
                      //write all the data from finalMail to the text file emails.txt! 
                      fs.writeFile("emails.txt", finalMail + "\n", function(err) {
                          if (err) {
                              console.log(err);
                          } else {
    
    
                          }
                      });
    
    
                  }
                  
                  //timer! delay
                  if (userids.length > index - 1) {
                      setTimeout(function() {
                          processNextUrl(index + 1);
                      }, timersetter);
                  }
              });
          }
      }
    
    
      //call the function(start!)
      processNextUrl(0);
    okey this is a lot to take in but should help you create your own scraper, I recommend starting a course on http://codecademy.com its free and you can learn javascript in a week, and you will understand how this script works and can make your own :)

    if you wanna start learning code I vouch for these websites:
    Code:
    http://www.codecademy.com/
    http://learncodethehardway.org/
    https://www.codeschool.com/
    http://www.w3schools.com/
    https://teamtreehouse.com/
    http://code.tutsplus.com/
    
    V0.5 this post will be updated based on facebook, Next week --> scraping UIDS from fan pages!
     
    • Thanks Thanks x 3
  2. handmadebots

    handmadebots Senior Member

    Joined:
    Nov 8, 2012
    Messages:
    933
    Likes Received:
    207
    Home Page:
    Good job man :)
     
    • Thanks Thanks x 1
  3. Nut-Nights

    Nut-Nights Jr. VIP Jr. VIP

    Joined:
    Jun 20, 2013
    Messages:
    3,920
    Likes Received:
    2,317
    Location:
    Hell
    Home Page:
    Nice tut but little bit confusing for me. let me try, will back if it works for me.
     
    • Thanks Thanks x 1
  4. Rockonaut

    Rockonaut Registered Member

    Joined:
    Feb 14, 2014
    Messages:
    76
    Likes Received:
    27
    sure! let me know how it goes and if I can help :)
     
  5. drifter48

    drifter48 Newbie

    Joined:
    Mar 10, 2012
    Messages:
    1
    Likes Received:
    0
    There's no way this works... graph API does not output "username" since version 1.0. Version 2.0 was introduced in May 2014

    So not sure how you're getting the username. Do explain.
     
  6. nanavlad

    nanavlad Jr. VIP Jr. VIP Premium Member

    Joined:
    Dec 2, 2009
    Messages:
    2,432
    Likes Received:
    899
    Gender:
    Male
    Occupation:
    SEO Consultant
    Location:
    Proxy Central
  7. Hendrik04

    Hendrik04 BANNED BANNED

    Joined:
    Nov 29, 2014
    Messages:
    69
    Likes Received:
    22
    Bullshit, this method not work anymore like mentioned before...