So you have a lot of UIDs? no problem today we are going to convert them to @facebook.com emails and use them for a custom audience! but we are going to make our own scraper for this!
Many of you have worked with HTML/CSS, some of you maybe worked with JAVA, C++ /C or any other languages for you this will be easy to pickup, if you don't have any experience it will take around a week to learn this
The final script we create:
This is javascript, well with some node.js. but what happens?
Facebook graph gives you user info based on their UID, for example:
output:
So when we add a UID behind graph.facebook.com we get an JSON file with their info! what we are intrested in are the usernames, if you add '@facebook.com' behind them you have their fb email ready to upload to the facebook powereditor.
but doing this by hand takes alot of time.. what we are going to make is a simple command line tool!
First you need to install some software:
http://www.sublimetext.com/
or any other text editor (no word etc.)
http://nodejs.org/
and
https://github.com/request/request
https://github.com/cheeriojs/cheerio
These final 2 can be installed with the command line, if you are new to the terminal read this book:
http://cli.learncodethehardway.org/book/ learn code the hardway is a great resource of info, check them out!
That was our setup, now lets start coding!
What happens first:
Here we call our modules like cheerio, request and declare variables, userids is interesting because we use a file to insert an array with the user ids!
The rest of the code is explained in the comments // <-- is a comment
okey this is a lot to take in but should help you create your own scraper, I recommend starting a course on http://codecademy.com its free and you can learn javascript in a week, and you will understand how this script works and can make your own
if you wanna start learning code I vouch for these websites:
V0.5 this post will be updated based on facebook, Next week --> scraping UIDS from fan pages!
Many of you have worked with HTML/CSS, some of you maybe worked with JAVA, C++ /C or any other languages for you this will be easy to pickup, if you don't have any experience it will take around a week to learn this
The final script we create:
Code:
var request = require('request'),
cheerio = require('cheerio'),
fs = require('fs');
var facebookurl = "http://graph.facebook.com/";
var email = "@facebook.com"
var finalMail = [];
var timersetter = Math.floor((Math.random() * 3000) + 300);
var userids = fs.readFileSync('uidsfiles.txt').toString().split("\n");
for (i in userids) {
function processNextUrl(index) {
var userid = userids[index];
totalurl = facebookurl + userid;
request(totalurl, function(error, response, html) {
if (!error && response.statusCode == 200) {
var obj = JSON.parse(html);
finalMail.push(obj.username + email);
console.log(obj);
console.log(finalMail.length + " done")
console.log(userids.length - finalMail.length + "to do");
// console.log(finalMail.length);
fs.writeFile("emails.txt", finalMail + "\n", function(err) {
if (err) {
console.log(err);
} else {
}
});
}
if (userids.length > index - 1) {
setTimeout(function() {
processNextUrl(index + 1);
}, timersetter);
}
});
}
}
processNextUrl(0);This is javascript, well with some node.js. but what happens?
Facebook graph gives you user info based on their UID, for example:
Code:
http://graph.facebook.com/100001525702202output:
Code:
{ "id": "100001525702202",
"first_name": "\u0421\u0430\u043d\u044f",
"gender": "male",
"last_name": "\u0421\u0435\u0440\u0433\u0435\u0435\u0432",
"link": "https://www.facebook.com/stuk.koles",
"locale": "ru_RU",
"name": "\u0421\u0430\u043d\u044f \u0421\u0435\u0440\u0433\u0435\u0435\u0432",
"username": "stuk.koles" }So when we add a UID behind graph.facebook.com we get an JSON file with their info! what we are intrested in are the usernames, if you add '@facebook.com' behind them you have their fb email ready to upload to the facebook powereditor.
but doing this by hand takes alot of time.. what we are going to make is a simple command line tool!
First you need to install some software:
http://www.sublimetext.com/
or any other text editor (no word etc.)
http://nodejs.org/
and
https://github.com/request/request
https://github.com/cheeriojs/cheerio
Code:
[COLOR=#333333][FONT=Consolas]npm install cheerio
[/FONT][/COLOR][COLOR=#333333][FONT=Consolas]npm install request[/FONT][/COLOR]These final 2 can be installed with the command line, if you are new to the terminal read this book:
http://cli.learncodethehardway.org/book/ learn code the hardway is a great resource of info, check them out!
That was our setup, now lets start coding!
What happens first:
Code:
var request = require('request'), cheerio = require('cheerio'),
fs = require('fs');
var facebookurl = "http://graph.facebook.com/";
var userdata = [];
var email = "@facebook.com"
var finalMail = [];
var timersetter = Math.floor((Math.random() * 3000) + 300);
var userids = fs.readFileSync('uidsfiles.txt').toString().split("\n");Here we call our modules like cheerio, request and declare variables, userids is interesting because we use a file to insert an array with the user ids!
The rest of the code is explained in the comments // <-- is a comment
Code:
var request = require('request'),
cheerio = require('cheerio'),
fs = require('fs');
var facebookurl = "http://graph.facebook.com/";
var userdata = [];
var email = "@facebook.com"
var finalMail = [];
//random time between 300 - 3000 ms
var timersetter = Math.floor((Math.random() * 3000) + 300);
//grap the UIDS from the file!
var userids = fs.readFileSync('uidsfiles.txt').toString().split("\n");
for (i in userids) {
function processNextUrl(index) {
var userid = userids[index];
//create an new URL with the next UID!
totalurl = facebookurl + userid;
request(totalurl, function(error, response, html) {
if (!error && response.statusCode == 200) {
//take the JSON data and make it an object!
var obj = JSON.parse(html);
//push the obj.username data + the @facebook.com to finalMail array!
finalMail.push(obj.username + email);
//print stuff out to the command line! just a reminder for myself to show whats happening!
console.log(obj);
console.log(finalMail.length + " done")
console.log(userids.length - finalMail.length + "to do");
//write all the data from finalMail to the text file emails.txt!
fs.writeFile("emails.txt", finalMail + "\n", function(err) {
if (err) {
console.log(err);
} else {
}
});
}
//timer! delay
if (userids.length > index - 1) {
setTimeout(function() {
processNextUrl(index + 1);
}, timersetter);
}
});
}
}
//call the function(start!)
processNextUrl(0);okey this is a lot to take in but should help you create your own scraper, I recommend starting a course on http://codecademy.com its free and you can learn javascript in a week, and you will understand how this script works and can make your own
if you wanna start learning code I vouch for these websites:
Code:
http://www.codecademy.com/
http://learncodethehardway.org/
https://www.codeschool.com/
http://www.w3schools.com/
https://teamtreehouse.com/
http://code.tutsplus.com/V0.5 this post will be updated based on facebook, Next week --> scraping UIDS from fan pages!
