BULLSHIT BULLSHIT BULLSHIT
First of all if, if you are going to talk about something in detail then atleast have the decency to educate yourself better on the subject instead of reinforcing the mob mentality BHW has towards social media filters when all your doing is spreading the same shit(ofcourse packaged in a nice format, but is still erroneous information like it or not) that is not true.Man, if I didn't call you out on this who would have done this? I guess no one.
Now I am going to tell you why in a very detailed constructive post with tons of pictures why your 3 types of filtering are all made up fairy tales bhw shit you just rehashed
There are 3 types of filtering networks have at there disposal. 1. Keyword filtering. 2. User based filtering (report spam buttons). 3. IP based filtering and browser filtering techniques.
MMMM... so you say that there are 3 types of filtering these networks have at their disposal, lol. Really? Who told you that? Did you learn all of this from bhw or did you actually do any research?Have you actually read any White Papers on the subject or does your knowledge on those supposely 3 types of filters come from reading hundreds of threads all saying the same thing from other uninformed bh'ers? Ever heard of the Facebook Immune System paper ?
Nah.... of course you didn't . Why would I even bother asking when the evidence is clear? How silly of me to think you did lol
If you?re account travels ?x? distance outside of these ?normal usage? statistics, a simple filter can be put in-place to either A. Send your account to a manual review, or B. Automatic ghosting, or account banning.
Oh wait... how could I forget, this thread also has one of those big famous bhw myth's , the "manual review". Geez, why do we need Random Forests, Naive Bayes, Regression ,Fuzzy Logic Matching Algorithms, and more Algorithms when there is manual review. I guess facebook is overpaying these engineers to use all kinds of Algorithms when you could just do the manual review route, so why build the Facebook Immune System to manage all that.
Funny how I never found any mentioning on their Immune System where there is "manual review" for accounts. I guess this "manual review" is really is an exclusive "BHW Urban Legend" lmao.
WARNING!!! LONG DETAILED CONSTRUCTIVE POST AHEAD WITH PICTURES
So before I start I would like to give credit to the first bhw member who posted http://www.blackhatworld.com/blackhat-seo/social-networking-sites/460856-must-read-8-page-report-facebook-spam-team-into-how-they-detect-fake-accounts.html almost a year ago but didn't get enough replies, however it did get 566 views but only 4 replies, and 9 thanks. From those 9 thanks one of them was from G-S-T who actually found this to be one of the best reads he had on that month, and even thanked the guy. Tacolypse thanked the guy too.That goes to show you the real NUGGETS are hidden somewhere in bhw and they don't have to be long to read(its all about the type of content that is in there not just the Quantity), in my honest opinion this is far from being a real NUGGET, but just a rehashed packed thread with infographics.
I guess sometimes quality is a lot better for the few members that appreciate Advanced Topics,even if its a simple link to a White Paper, than a long guide with a bunch of erroneous information with pictures from BHW that doesn't really bring anything new to these thirsty guys seeking for something new to read.For everyone else these guides are a wonderful ways to read your rehash information and for the rest it would make extra money rehashing this into a clickbank product.
Now with my constructive post....
So first lets go over some basics, as posted on the Research Paper.
This is the figure 1 - The adversarial cycle.
A quick explanation goes like this, the Attacker(can be a spammer, hacker, creeper) attacks the system, when the first Attack is detected the System does nothing to combat it. What it actually does is constructs a new training model , and collects as much data as it can in order to better classify this new threat to the Graph. So while the attacker might not know this, he is actually contributing to his own demise. By doing the same type of attack and not changing much, your training a new model where it can easily assemble a very good training model to defend itself once its ready. On the defense stage, this is where the attackers attacks become immune to the system. Either the attacker adapts and changes his tactics, before the Immune System mutates(this means it basically adapts before you to change its defense system before you even plan your next attack) or the cycle ends with the defenders winning the game.
Its a never-ending cat and mouse game played by Facebook and its Attackers(again, spammers, creepers, hackers etc you get the point) . Only the strongest survive this cycle for a long long time, the rest die off.
So if you can get something out of this, just get this. This is a self-learning system, that actually uses data and statistical models to adapt to the environment without much human input. In other words, it can easily have a defense for thousands of different spammers based on their signature of attack, or a big botnet like koobface that uses other compromised computers to do its dirty work, or just annoying fuckers who send those chain letters.It can fight all of those.
Ghosting and Mutate
The phases the attacker controls are the Attacks, and the Detection phases. If you fail to detect before the system mutates, that makes your Detection phase shorter and useless which can lead to your Attacks shorter too. That is the ultimate goal of this system, prevent your detection rate, and your attacks and make their defend and mutate phases longer. That is why they have Ghosting, as a self-defense mechanism that outputs obscure messages, and mutate features like requiring a different IP(or restricting activities per IP basis) which makes it more expensive to keep playing this game for the Attackers.
Actual Decision Making - Not manual review
However, in times of emergency they do have to do damage control. Because it takes time to train a new model and build up the defenses while the attack is happening. It is sometimes better to focus more on saving 98% of 100k users than saving 99% of 1k users, so is not viable to wait longer until a more accurate classifier is made in order to combat the attack. That is where human intervention occurs, especially when you have those viral pages going on. Those are manual reviews that are done in order to eliminate the threat when it becomes viral.
Its time for the fun part, the 4 Main Components of the Facebook Immune System
4 Main Components of Facebook Immune System
Now pay attention to where it says
"
In addition to responding quickly, it is important to target features
that are difficult for the attacker to detect (Defense) and
change (Mutate). This differs from traditional machine-learning
where the features are chosen solely on how strongly they improve
the accuracy of the classifier.
"
This is key to understanding why there IP, Cookies, IP:ACCT ratio are what you shouldn't just solely focus on. It clearly says there that it is important to target shit that you ain't looking at. So to make it a lot easier for all of you to understand, if you only thing that facebook can track you by your proxies, regular and flash cookies, comments(you know the main shit people say when you get detected)then you are very misinformed because no one is talking about the other 495 other feature datasets that we ain't tracking, unless that is you are part of the .0001% that tracks more than just the main 4-5 feature datasets and make your own models.
Just understand this IP and browser user-agents filters, Comment filters, and User based filtering are not FILTERS, they are Feature DataSets, WHICH IS A VERY BIG DIFFERENCE OF WHAT ZENOGLITCH IS CALLING THEM
NOTE: IP filtering is not a main filter like zenoGlitch claims, but one of the many 500 Feature sets
500 Feature Sets
This is a picture of the whole Immune System with all of its components.
Figure 3 - High-Level design of the Immune System
Policies are what you would call your filters, these filters require data which comes from the feature datasets, they use classifiers to do their algorithm modeling for them providing values that are sent back to the policy manager to do its decision. Inside the Feature Data providers 3 loops keeps the data up-to-date with counters there are 3 loops, inner, middle, and outer loop(More on that later).
Policy Layer
Policy Layer-2
There are 2 types of policies, business and logic . The main difference is that business doesn't require any trained data and doesn't mutate. This basically means that its a hard-coded rule, the logic one uses trained models and is always-mutating. It is important to know this, because every single featuredata set can actually trigger another set of policy rules. In other words, lets say you do 1 comment on 1 fb acct and you do have logged into that acct from that ip.
If you are only tracking your comments per acct, and you get banned then you would automatically think it was because you did x comments. That is the wrong way to think about it, because it could have been from other feature datasets that triggered that banned. Which means that IP, Comments to IP ratio aren't as important as you think because those are only part of certain policies that use those feature datasets as params to evaluate the outcome.Remember this exert from the previous quote"where the features are chosen solely on how strongly they improve
the accuracy of the classifier." well this means that basically where IP, IP:Acct , Comment:ACCT ratio were very important at first, now they matter less because other feature sets are given more weight than those feature datasets when you are using normal rates for those feature datasets. Just remember again there are over 500 Feature Datasets
Design - ClassifyScore not TrustScore
This is actually called the ClassifyScore , and it is not for just Accounts. This is used for every classifier
It?s not always as simple as just figuring out the what and the when. Sometimes gaming the popular feeds have more complex algorithms that take into consideration the ?trust score? of the account, the relevent content on the account, the keywords used, and even manual reviews for popular content in rare cases.
Classify Score - Not Trust Score
Classifier in real-time
Classifier in real-time-2
From those 2 paragraphs what you need to get out of this is that there are mainly 2 types of classifiers, real-time and not real-time. The real-time are classifiers that models and classify feature data sets in real-time. For example, for the facebook chat if you are sending the same message over and over, this is an example of real-time where its instant classification of a spam message. The non-live require time to read more data and understand what is going before making a decision. That is why it takes time for you to get banned, because it completely analyzes all of the feature datasets before making a hard decision. YOu ever notice why you don't get banned right away after doing x actions, is because evaluation is in process through these modeling is not instant.
Just so you know they compile test cases of the entire world(all of it facebook) to build new models and test certain certain models. Ask yourself are you doing that?
Alongside of keyword detections there will be string analyzers that search for duplicate match strings over ?x? volume. So if exact match string exceed acceptable duplicatent comment rate in ?y? ammount of time then flag/ban/delete/ send this account to a queue for manual review.
They use Fuzzy Logic Matching Algorithms they don't need to do what you say they do.
Ever heard of
-N-gram
- Q-gram-based Algorithms
- Levenstein
Yea well... it clearly says they use Fuzzy logic Matching Algorithms
Fuzzy Logic Matching
Now for creating the rules you use FXL, now it is important to know this. The expressions use extensive use of subtrees and memoization techniques, this is not some "if this x then y" bullshit like OP is saying. Ever heard of Ensemble learning?
Expressions as trees - memoization, subtrees. This is mostly just a scripting language they created to create the policies and the classifiers.
Feature Data Providers
Feature Loops
Loops - Inner_Middle_Outer
Now the important stuff to remember is that those feature datasets need to be updated with counters, that is why they have 3 counters.
The inner loop keeps track of simple counters on the feature datasets(like how many times user has done, comments, posts,likes, etc), the outer does a little more computing which computes expressions for matching certain ip's and urls. The outer loop is a way more complicated which does something different, it basically keeps track of the number data from across the network. So in other words, it actually can keep track of the number of times x url has been posted across the whole network, or it can match any message from all the users on facebook to see how many times certain comments are similar to each other.
So remember, innerloop is to keep track of the feature datasets like simple counters, number of likes, comments, posts, etc
Middleloop keeps track of expressions using these feature data sets, like "if x comment = bla bla bla" but using fuzzy logic matching algorithms, especially N-grams.
Outerloop, keeps track of the overall featuredata sets across the whole network to spot similarities across the network . This is to prevent mostly phishing scams, viral pages, malware infections that take over the network virally.
There you have it folks, a very simple breakdown of how the Facebook Immune System works, not just some fairy tale bullshit that the OP is rehashing and packaging for you. You see where there is no mention of "manual review quee", there is no 3 filters based on IP, commenting, and user-based filters.
Remember, the biggest advantage Facebook has over you is user-feedback and Global Data that it uses for Data Modeling, most people don't do that.
If you have read this whole thing up to now. Here is a trick I will show you as a token of appreciation for reading this whole damn long post,this again was posted on bhw over 2 years ago. I can't find the thread but I can tell you it got buried down under so many threads like this.
If you want to create an account on facebook that is captcha free after acct creation(without being pva) then when you go and create a facebook account, CHOOSE AN EMAIL THAT IS FROM A UNIVERSITY, even if you don't own it. Here is how it works.
say you put in the email something like this
[email protected], when you finish creating that account. Change your email to the one you were going to use, and resend the verification email. Guess what? You will be captcha free on that account. You see how changing the email can make a difference. That is how Facebook works, they check a lot of your data and I can tell you a university Email is more valuable than a yahoo email when doing an acct creation. Trust me they do look into the type of email you are using. So if they are looking at the email used and they even classify different email providers as spammy, and good then don't you think they would use other Feature DataSets as a way to determine if you are legit or a spammer besides jsut IP, cookies, and x comment,likes, post ratio. Think about it.
