The Future of Ad Blocking

Discussion in 'BlackHat Lounge' started by Asif WILSON Khan, May 3, 2017.

  1. Asif WILSON Khan

    Asif WILSON Khan OG Blue Tick Exec VIP Jr. VIP

    Nov 10, 2012
    Likes Received:
    Fun Lovin' Criminal
    Home Page:
    There’s an ongoing arms race between ad blockers and websites — more and more sites either try to sneak their ads through or force users to disable ad blockers. Most previous discussions have assumed that this is a cat-and-mouse game that will escalate indefinitely. But in a new paper, accompanied by proof-of-concept code, we challenge this claim. We believe that due to the architecture of web browsers, there’s an inherent asymmetry that favors users and ad blockers. We have devised and prototyped several ad blocking techniques that work radically differently from current ones. We don’t claim to have created an undefeatable ad blocker, but we identify an evolving combination of technical and legal factors that will determine the “end game” of the arms race.

    [​IMG]Our project began last summer when Facebook announced that it had made ads look just like regular posts, and hence impossible to block. Indeed, Adblock Plus and other mainstream ad blockers have been ineffective on Facebook ever since. But Facebook’s human users have to be able to tell ads apart because of laws against misleading advertising. So we built a tool that detects Facebook ads the same way a human would, deliberately ignoring hidden HTML markup that can be obfuscated. (Adblock Plus, on the other hand, is designed to be able to examine only the markup of web pages and not the content.) Our Chrome extension has several thousand users and continues to be effective.

    We’ve built on this early success. Laws against misleading advertising apply not just on Facebook, but everywhere on the web. Due to these laws and in response to public-relations pressure, the online ad industry has developed robust self-regulation that standardizes the disclosure of ads across the web. Once again, ad blockers can exploit this, and that’s what our perceptual ad blocker does. [1]

    The second prong of an ad blocking strategy is to deal with websites that try to detect (and in turn block) ad blockers. To do this, we introduce the idea of stealth. The only way that a script on a web page can “see” what’s drawn on the screen is to ask the user’s browser to describe it. But ad blocking extensions can control the browser! Not perfectly, but well enough to get the browser to convincingly lie to the web page script about the very existence of the ad blocker. Our proof-of-concept stealthy ad blocker successfully blocked ads and hid its existence on all 50 websites we looked at that are known to deploy anti-adblocking scripts. Finally, we have also investigated ways to detect and block the ad blocking detection scripts themselves. We found that this is feasible but cumbersome; at any rate, it is unnecessary as long as stealthy ad blocking is successful.

    The details of all these techniques get extremely messy, and we encourage the interested reader to check out the paper. While some of the details may change, we’re confident of our long-term assessment. That’s because our techniques are all based on sound computer security principles and because we’ve devised a state diagram that describes the possible actions of websites and ad blockers, bringing much-needed clarity to the analysis and helping ensure that there won’t be completely new techniques coming out of left field in the future.

    There’s a final wrinkle: the publishing and advertising industries have put forth a number of creative reasons to argue that ad blockers violate the law, and indeed Adblock Plus has been sued several times (without success so far). We carefully analyzed four bodies of law that may support such legal claims, and conclude that the law does not stand in the way of deploying sophisticated ad blocking techniques. [2] That said, we acknowledge that the ethics of ad blocking are far from clear cut. Our research is about what can be done and not what should be done; we look forward to participating in the ethical debate.

    [1] To avoid taking sides on the ethics of ad blocking, we have deliberately stopped short of making our proof-of-concept tool fully functional — it is configured to detect ads but not actually block them.

    [2] One of the authors is cyberlaw expert Jonathan Mayer.


    Have These Researchers Created An Unbeatable Ad-Blocking Technology?

    Academics have developed a set of approaches that not only identify web ads but deter anti-blocking strategies.

    Here’s the thing about an ad: If you can’t recognize it, it’s worth nothing to the advertiser. That’s the fatal flaw with web-based ads. No matter how much ad technology evades ad-blocking software by disguising itself, it still has to be recognizable to a user and potentially clickable.

    Researchers at Princeton and Stanford believe they have shown how to end the escalating blocker/anti-blocker battle as a result of that crucial point, and in favor of user choice. While a “war to win our eyeballs” sounds like the theme of a Guillermo del Toro film, it describes the interplay between advertisers (and ad-technology companies) and the visitors who reject the panoply of tracking techniques and page bloat that come with current online ads.

    Some sites go beyond just trying to route around blocking techniques used by Ghostery, AdBlock Plus, and others by showing a scolding message when they detect blocking action in use. A visitor often has to disable an ad blocker or add a rules exception to proceed to a site. But Princeton and Stanford’s academics have determined it’s possible to identify ads with an extremely high degree of reliability without using any of the current ad-blocking tricks of identifying underlying page elements, domains, and the like, and also block counter-defenses from sites and adtech companies.

    In a paper currently in draft form, the authors detail an interlocking set of theory, code, and legal reasoning about the state of ad blocking and the response by ad networks and site publishers. It’s been assumed that the blocking and anti-blocking war would escalate indefinitely, with battles fought as a series of measures and countermeasures. The researchers lay out the case that browser users and browser makers have the upper hand, and that in any given skirmish, publishers will quickly lose.

    The Telltale Signs Of Advertising
    Instead of looking at network and code, the proof of concept the authors first deployed as a Chrome plug-in–which identifies ads on Facebook–uses computer vision, optical-character recognition of text rendered as images, and other cues. It allows ads to load and scripts to run, at which point it can determine what on a page is an ad.

    To discourage robots from automatically filling them out, text-based CAPTCHAs became ever more baroque to avoid scripts puzzling out the results, to the point where they frustrated many users as well as the bots. That can’t work with ads; it even stopped working with CAPTCHAs, as scammers adopted deep-learning computer vision techniques. “So long as advertisements, even malicious advertisements, are recognizable by users, you should be able to use these techniques to find them,” says Grant Storey, a Princeton undergraduate in computer science who coauthored the paper with Arvind Narayanan and Dillon Reisman of Princeton and Jonathan Mayer of Stanford. (Mayer is currently at work in the FCC’s enforcement bureau as chief technologist.)

    Blocking ads on Facebook.
    Their approach relies in part on legitimate advertisers, ad networks, and publishers complying with U.S. regulations and with guidelines for industry self-monitoring. Reputable ads have labels and other attributes that make them stand out. It might be subtle to a user, but it’s obvious to a trained machine-learning system. (Other countries vary in their practices, though some have even stricter laws and industry self-monitoring.)

    As the researchers note, “In order to defeat a filter list [such as is used by conventional ad blockers], all that is required is moving an advertisement to a different URL; in order to defeat a perceptual ad blocker, an entirely new ad disclosure standard must be approved.” The researchers limited their testing to ads on Facebook pages and ads that comply with regulations and industry practice. “For this paper, our focus was on this well-behaved universe, where there are certain sort of norms that are being followed,” Storey says.

    The researchers’ system is modular and adaptable, and could be trained to recognize unlabeled ads, although the researchers have found that over time more advertising on more sites has proper labels and disclosure. Their framework doesn’t encompass “malvertising,” or the delivery of malware via ads. Anti-malware, Google Safe Browsing, and other software and services better handle that separate from identifying them as ads. Nor does it block the trackers that are often part of ad serving, but are a concern because of privacy issues rather than than visual interaction.

    [Photo: Flickr user Phil Roeder]
    Uncanny Accuracy
    In their testing, the Facebook extension, in the field for several months, matched 50 out of 50 ads, including those in both the news feed and sidebars. The four researchers also report they saw no false negatives or positives in their personal use over six months.

    On the broader web, they tested a module that looks for disclosures under the AdChoices program, used in North America and Europe, and which the papers’ authors found was used in over 60% of ads in a sample of 183 ads from top news websites. Their AdChoices module correctly labeled over 95% of AdChoices ads from 100 sites randomly selected from the top 500 news sites.

    The researchers’ technology could create a beneficial feedback loop, too, as users who might employ ad-detection software could complain to advertisers, sites, ad networks, state attorneys general, trade groups, and the FTC about commercial messages that were identifiable as out of compliance with regulations and industry guidelines. (In fact, this approach could be automated by nonprofit and governmental consumer-protection groups to identify out-of-compliance ads.)

    On top of ad identification, the paper offers a further step in dampening the powder on the adtech side of this battle. Because the technology the researchers tested comes in the form of a browser extension, it has privileges that extend far beyond what JavaScript code can do in a browser. That allows developers to turn a loaded web page into a kind of “brain in a jar,” which they label a “rootkit,” because of its advantageous position in the browser. The researchers can use this fact to prevent anti-blocking software from determining whether an ad blocker is in use, even if the software detects that it’s been sandboxed.

    And, with a similar approach, the researchers tested whether it’s possible to create a differential examination of a page, by loading it once and applying ad blocking and then loading a “shadow” version that executes all page-modifying JavaScript code. The two versions could be compared to see if anti-ad blocking messages or changes took place. By figuring out what elements are being tracked, the extension could return responses that the publisher would expect only from a page showing its ads, thereby allowing it to block ads without detection. (The authors didn’t implement this in code, but tested whether it would be effective.)

    Blocking AdChoices-compliant ads.
    These techniques, and another exploration into blocking the execution of anti-blocking code altogether, raise ethical concerns that are addressed briefly in the paper, because such tools could be used in advertising fraud, a large industry in which automated scripts attempt to rack up page views and perform clicks while appearing to be legitimate actions by humans.

    The research might offer more insight to fraudsters in preventing detection by using extensions, but, Storey notes, “there are still other ways to detect the ad-fraud bot that should available” and these techniques don’t work for fraud systems that load in a browser. The researchers also omitted a few details to prevent releasing full details on their technique.

    The brain-a-in-jar method could be escalated further if browser makers go further and either provide deeper access for extension creators or build in ad blocking directly. Google reportedly is considering changes to Chrome that would prevent certain kinds of irritating ads from loading or bar all ads from loading on pages that use any of those forms of irritating ads.

    The only way to win most wars is to avoid conflict in the first place. As web-ad revenue has slipped away to Facebook, Twitter, and mobile apps, among other places, publishers have developed adtech or signed up with networks that offer it. That’s led to heavier use of invasive techniques such as pop-up ads with hard-to-click Xs to close and auto-play video, as well as large downloads for the web code to support them.

    JPMorgan Chase recently discovered that automated advertising on 400,000 sites brought clicks only from 12,000. It winnowed that list to 5,000 handpicked sites and saw no overall change in results. That would indicate that aggressive techniques to deliver ads to users aren’t working for advertisers, either.

    Princeton and Stanford’s research, combined with results like those from Chase, might force publishers to rethink ad approaches entirely. That could lead them to back out of the blocking/anti-blocking situation, finding a way to attract users into viewing well-behaved marketing and leaving the tricks behind.


    • Thanks Thanks x 2
  2. HatArrows

    HatArrows Regular Member

    Jun 28, 2013
    Likes Received:
    Data Mines
    What i have seen lately on facebook or some other mainstream sites , the ad space or even the trends has become somewhat a political space, rather than showing things that matches our interests , it looks like an intrusion in our mind. The content on ads should also be marked as political or commercial
  3. pressrelease

    pressrelease Power Member

    Jan 6, 2016
    Likes Received:
    Long read, but to bypass adblock disable notice , i use JavaScript switch plugin to turn it on or off, as moatly these adblock disable message caused by JavaScript calls, though turning off script affect sites functions to load.
  4. sturose

    sturose Elite Member

    Nov 6, 2013
    Likes Received:
    It's a battle that has been raging for a long time now, content providers and ad servers vs adblockers. I don't believe either side will ever win because if one creates a new method to serve ads or block ads the other team will find some way to counteract what they have done. Yes they may make it difficult but I don't think it will ever be impossible to crack their code.

    I do find it annoying when I visit a site and I am met with a huge pop up telling me to disable my adblocker, I don't have a problem with doing this but FFS ask me nicely and I might!!! Demand that I do and I'll hit the back button.

    I find it quite ironic that many of us on here block other sites ads using adblock plus or similar but want other internet users to view and click our ads.

    Me included, I use adblock plus and most sites ads are blocked.:confused:
  5. soulcollector

    soulcollector Senior Member

    May 10, 2014
    Likes Received:
    I block crappy ads, but sometimes I just like to see what the competition is feeding the mindless droves of slaves with credit cards ... I hate ads from companies that have to brag about low prices. These companies are generally making so much margin that "low prices" translates into "shop here, I make 35% margin even with the sales going on!" - I will not shop there because of the ad.
  6. zionbar

    zionbar Jr. VIP Jr. VIP

    Apr 6, 2015
    Likes Received:
    Sunny Florida
    I don't get the big websites like youtube, they should block adblock users 100% of the times.
    I use adblock myself but it fucks up my revenue, and as long as I can use it of course I will but they shouldnt give people the option to block ads I mean you're watching videos and getting content FREE at least watch some freakin ads aight ? :D