Author Archives: Bruce Schneier

How Apple’s "Find My" Feature Works

Matthew Green intelligently speculates about how Apple's new "Find My" feature works.

If you haven't already been inspired by the description above, let me phrase the question you ought to be asking: how is this system going to avoid being a massive privacy nightmare?

Let me count the concerns:

  • If your device is constantly emitting a BLE signal that uniquely identifies it, the whole world is going to have (yet another) way to track you. Marketers already use WiFi and Bluetooth MAC addresses to do this: Find My could create yet another tracking channel.

  • It also exposes the phones who are doing the tracking. These people are now going to be sending their current location to Apple (which they may or may not already be doing). Now they'll also be potentially sharing this information with strangers who "lose" their devices. That could go badly.

  • Scammers might also run active attacks in which they fake the location of your device. While this seems unlikely, people will always surprise you.

The good news is that Apple claims that their system actually does provide strong privacy, and that it accomplishes this using clever cryptography. But as is typical, they've declined to give out the details how they're going to do it. Andy Greenberg talked me through an incomplete technical description that Apple provided to Wired, so that provides many hints. Unfortunately, what Apple provided still leaves huge gaps. It's into those gaps that I'm going to fill in my best guess for what Apple is actually doing.

Hacking Hardware Security Modules

Security researchers Gabriel Campana and Jean-Baptiste Bédrune are giving a hardware security module (HSM) talk at BlackHat in August:

This highly technical presentation targets an HSM manufactured by a vendor whose solutions are usually found in major banks and large cloud service providers. It will demonstrate several attack paths, some of them allowing unauthenticated attackers to take full control of the HSM. The presented attacks allow retrieving all HSM secrets remotely, including cryptographic keys and administrator credentials. Finally, we exploit a cryptographic bug in the firmware signature verification to upload a modified firmware to the HSM. This firmware includes a persistent backdoor that survives a firmware update.

They have an academic paper in French, and a presentation of the work. Here's a summary in English.

There were plenty of technical challenges to solve along the way, in what was clearly a thorough and professional piece of vulnerability research:

  1. They started by using legitimate SDK access to their test HSM to upload a firmware module that would give them a shell inside the HSM. Note that this SDK access was used to discover the attacks, but is not necessary to exploit them.

  2. They then used the shell to run a fuzzer on the internal implementation of PKCS#11 commands to find reliable, exploitable buffer overflows.

  3. They checked they could exploit these buffer overflows from outside the HSM, i.e. by just calling the PKCS#11 driver from the host machine

  4. They then wrote a payload that would override access control and, via another issue in the HSM, allow them to upload arbitrary (unsigned) firmware. It's important to note that this backdoor is persistent ­ a subsequent update will not fix it.

  5. They then wrote a module that would dump all the HSM secrets, and uploaded it to the HSM.

Risks of Password Managers

Stuart Schechter writes about the security risks of using a password manager. It's a good piece, and nicely discusses the trade-offs around password managers: which one to choose, which passwords to store in it, and so on.

My own Password Safe is mentioned. My particular choices about security and risk is to only store passwords on my computer -- not on my phone -- and not to put anything in the cloud. In my way of thinking, that reduces the risks of a password manager considerably. Yes, there are losses in convenience.

Maciej Cegłowski on Privacy in the Information Age

Maciej Cegłowski has a really good essay explaining how to think about privacy today:

For the purposes of this essay, I'll call it "ambient privacy" -- the understanding that there is value in having our everyday interactions with one another remain outside the reach of monitoring, and that the small details of our daily lives should pass by unremembered. What we do at home, work, church, school, or in our leisure time does not belong in a permanent record. Not every conversation needs to be a deposition.

Until recently, ambient privacy was a simple fact of life. Recording something for posterity required making special arrangements, and most of our shared experience of the past was filtered through the attenuating haze of human memory. Even police states like East Germany, where one in seven citizens was an informer, were not able to keep tabs on their entire population. Today computers have given us that power. Authoritarian states like China and Saudi Arabia are using this newfound capacity as a tool of social control. Here in the United States, we're using it to show ads. But the infrastructure of total surveillance is everywhere the same, and everywhere being deployed at scale.

Ambient privacy is not a property of people, or of their data, but of the world around us. Just like you can't drop out of the oil economy by refusing to drive a car, you can't opt out of the surveillance economy by forswearing technology (and for many people, that choice is not an option). While there may be worthy reasons to take your life off the grid, the infrastructure will go up around you whether you use it or not.

Because our laws frame privacy as an individual right, we don't have a mechanism for deciding whether we want to live in a surveillance society. Congress has remained silent on the matter, with both parties content to watch Silicon Valley make up its own rules. The large tech companies point to our willing use of their services as proof that people don't really care about their privacy. But this is like arguing that inmates are happy to be in jail because they use the prison library. Confronted with the reality of a monitored world, people make the rational decision to make the best of it.

That is not consent.

Ambient privacy is particularly hard to protect where it extends into social and public spaces outside the reach of privacy law. If I'm subjected to facial recognition at the airport, or tagged on social media at a little league game, or my public library installs an always-on Alexa microphone, no one is violating my legal rights. But a portion of my life has been brought under the magnifying glass of software. Even if the data harvested from me is anonymized in strict conformity with the most fashionable data protection laws, I've lost something by the fact of being monitored.

He's not the first person to talk about privacy as a societal property, or to use pollution metaphors. But his framing is really cogent. And "ambient privacy" is new -- and a good phrasing.

Data, Surveillance, and the AI Arms Race

According to foreign policy experts and the defense establishment, the United States is caught in an artificial intelligence arms race with China -- one with serious implications for national security. The conventional version of this story suggests that the United States is at a disadvantage because of self-imposed restraints on the collection of data and the privacy of its citizens, while China, an unrestrained surveillance state, is at an advantage. In this vision, the data that China collects will be fed into its systems, leading to more powerful AI with capabilities we can only imagine today. Since Western countries can't or won't reap such a comprehensive harvest of data from their citizens, China will win the AI arms race and dominate the next century.

This idea makes for a compelling narrative, especially for those trying to justify surveillance -- whether government- or corporate-run. But it ignores some fundamental realities about how AI works and how AI research is conducted.

Thanks to advances in machine learning, AI has flipped from theoretical to practical in recent years, and successes dominate public understanding of how it works. Machine learning systems can now diagnose pneumonia from X-rays, play the games of go and poker, and read human lips, all better than humans. They're increasingly watching surveillance video. They are at the core of self-driving car technology and are playing roles in both intelligence-gathering and military operations. These systems monitor our networks to detect intrusions and look for spam and malware in our email.

And it's true that there are differences in the way each country collects data. The United States pioneered "surveillance capitalism," to use the Harvard University professor Shoshana Zuboff's term, where data about the population is collected by hundreds of large and small companies for corporate advantage -- and mutually shared or sold for profit The state picks up on that data, in cases such as the Centers for Disease Control and Prevention's use of Google search data to map epidemics and evidence shared by alleged criminals on Facebook, but it isn't the primary user.

China, on the other hand, is far more centralized. Internet companies collect the same sort of data, but it is shared with the government, combined with government-collected data, and used for social control. Every Chinese citizen has a national ID number that is demanded by most services and allows data to easily be tied together. In the western region of Xinjiang, ubiquitous surveillance is used to oppress the Uighur ethnic minority -- although at this point there is still a lot of human labor making it all work. Everyone expects that this is a test bed for the entire country.

Data is increasingly becoming a part of control for the Chinese government. While many of these plans are aspirational at the moment -- there isn't, as some have claimed, a single "social credit score," but instead future plans to link up a wide variety of systems -- data collection is universally pushed as essential to the future of Chinese AI. One executive at search firm Baidu predicted that the country's connected population will provide them with the raw data necessary to become the world's preeminent tech power. China's official goal is to become the world AI leader by 2030, aided in part by all of this massive data collection and correlation.

This all sounds impressive, but turning massive databases into AI capabilities doesn't match technological reality. Current machine learning techniques aren't all that sophisticated. All modern AI systems follow the same basic methods. Using lots of computing power, different machine learning models are tried, altered, and tried again. These systems use a large amount of data (the training set) and an evaluation function to distinguish between those models and variations that work well and those that work less well. After trying a lot of models and variations, the system picks the one that works best. This iterative improvement continues even after the system has been fielded and is in use.

So, for example, a deep learning system trying to do facial recognition will have multiple layers (hence the notion of "deep") trying to do different parts of the facial recognition task. One layer will try to find features in the raw data of a picture that will help find a face, such as changes in color that will indicate an edge. The next layer might try to combine these lower layers into features like shapes, looking for round shapes inside of ovals that indicate eyes on a face. The different layers will try different features and will be compared by the evaluation function until the one that is able to give the best results is found, in a process that is only slightly more refined than trial and error.

Large data sets are essential to making this work, but that doesn't mean that more data is automatically better or that the system with the most data is automatically the best system. Train a facial recognition algorithm on a set that contains only faces of white men, and the algorithm will have trouble with any other kind of face. Use an evaluation function that is based on historical decisions, and any past bias is learned by the algorithm. For example, mortgage loan algorithms trained on historic decisions of human loan officers have been found to implement redlining. Similarly, hiring algorithms trained on historical data manifest the same sexism as human staff often have. Scientists are constantly learning about how to train machine learning systems, and while throwing a large amount of data and computing power at the problem can work, more subtle techniques are often more successful. All data isn't created equal, and for effective machine learning, data has to be both relevant and diverse in the right ways.

Future research advances in machine learning are focused on two areas. The first is in enhancing how these systems distinguish between variations of an algorithm. As different versions of an algorithm are run over the training data, there needs to be some way of deciding which version is "better." These evaluation functions need to balance the recognition of an improvement with not over-fitting to the particular training data. Getting functions that can automatically and accurately distinguish between two algorithms based on minor differences in the outputs is an art form that no amount of data can improve.

The second is in the machine learning algorithms themselves. While much of machine learning depends on trying different variations of an algorithm on large amounts of data to see which is most successful, the initial formulation of the algorithm is still vitally important. The way the algorithms interact, the types of variations attempted, and the mechanisms used to test and redirect the algorithms are all areas of active research. (An overview of some of this work can be found here; even trying to limit the research to 20 papers oversimplifies the work being done in the field.) None of these problems can be solved by throwing more data at the problem.

The British AI company DeepMind's success in teaching a computer to play the Chinese board game go is illustrative. Its AlphaGo computer program became a grandmaster in two steps. First, it was fed some enormous number of human-played games. Then, the game played itself an enormous number of times, improving its own play along the way. In 2016, AlphaGo beat the grandmaster Lee Sedol four games to one.

While the training data in this case, the human-played games, was valuable, even more important was the machine learning algorithm used and the function that evaluated the relative merits of different game positions. Just one year later, DeepMind was back with a follow-on system: AlphaZero. This go-playing computer dispensed entirely with the human-played games and just learned by playing against itself over and over again. It plays like an alien. (It also became a grandmaster in chess and shogi.)

These are abstract games, so it makes sense that a more abstract training process works well. But even something as visceral as facial recognition needs more than just a huge database of identified faces in order to work successfully. It needs the ability to separate a face from the background in a two-dimensional photo or video and to recognize the same face in spite of changes in angle, lighting, or shadows. Just adding more data may help, but not nearly as much as added research into what to do with the data once we have it.

Meanwhile, foreign-policy and defense experts are talking about AI as if it were the next nuclear arms race, with the country that figures it out best or first becoming the dominant superpower for the next century. But that didn't happen with nuclear weapons, despite research only being conducted by governments and in secret. It certainly won't happen with AI, no matter how much data different nations or companies scoop up.

It is true that China is investing a lot of money into artificial intelligence research: The Chinese government believes this will allow it to leapfrog other countries (and companies in those countries) and become a major force in this new and transformative area of computing -- and it may be right. On the other hand, much of this seems to be a wasteful boondoggle. Slapping "AI" on pretty much anything is how to get funding. The Chinese Ministry of Education, for instance, promises to produce "50 world-class AI textbooks," with no explanation of what that means.

In the democratic world, the government is neither the leading researcher nor the leading consumer of AI technologies. AI research is much more decentralized and academic, and it is conducted primarily in the public eye. Research teams keep their training data and models proprietary but freely publish their machine learning algorithms. If you wanted to work on machine learning right now, you could download Microsoft's Cognitive Toolkit, Google's Tensorflow, or Facebook's Pytorch. These aren't toy systems; these are the state-of-the art machine learning platforms.

AI is not analogous to the big science projects of the previous century that brought us the atom bomb and the moon landing. AI is a science that can be conducted by many different groups with a variety of different resources, making it closer to computer design than the space race or nuclear competition. It doesn't take a massive government-funded lab for AI research, nor the secrecy of the Manhattan Project. The research conducted in the open science literature will trump research done in secret because of the benefits of collaboration and the free exchange of ideas.

While the United States should certainly increase funding for AI research, it should continue to treat it as an open scientific endeavor. Surveillance is not justified by the needs of machine learning, and real progress in AI doesn't need it.

This essay was written with Jim Waldo, and previously appeared in Foreign Policy.

Computers and Video Surveillance

It used to be that surveillance cameras were passive. Maybe they just recorded, and no one looked at the video unless they needed to. Maybe a bored guard watched a dozen different screens, scanning for something interesting. In either case, the video was only stored for a few days because storage was expensive.

Increasingly, none of that is true. Recent developments in video analytics -- fueled by artificial intelligence techniques like machine learning -- enable computers to watch and understand surveillance videos with human-like discernment. Identification technologies make it easier to automatically figure out who is in the videos. And finally, the cameras themselves have become cheaper, more ubiquitous, and much better; cameras mounted on drones can effectively watch an entire city. Computers can watch all the video without human issues like distraction, fatigue, training, or needing to be paid. The result is a level of surveillance that was impossible just a few years ago.

An ACLU report published Thursday called "the Dawn of Robot Surveillance" says AI-aided video surveillance "won't just record us, but will also make judgments about us based on their understanding of our actions, emotions, skin color, clothing, voice, and more. These automated 'video analytics' technologies threaten to fundamentally change the nature of surveillance."

Let's take the technologies one at a time. First: video analytics. Computers are getting better at recognizing what's going on in a video. Detecting when a person or vehicle enters a forbidden area is easy. Modern systems can alarm when someone is walking in the wrong direction -- going in through an exit-only corridor, for example. They can count people or cars. They can detect when luggage is left unattended, or when previously unattended luggage is picked up and removed. They can detect when someone is loitering in an area, is lying down, or is running. Increasingly, they can detect particular actions by people. Amazon's cashier-less stores rely on video analytics to figure out when someone picks an item off a shelf and doesn't put it back.

More than identifying actions, video analytics allow computers to understand what's going on in a video: They can flag people based on their clothing or behavior, identify people's emotions through body language and behavior, and find people who are acting "unusual" based on everyone else around them. Those same Amazon in-store cameras can analyze customer sentiment. Other systems can describe what's happening in a video scene.

Computers can also identify people. AIs are getting better at identifying people in those videos. Facial recognition technology is improving all the time, made easier by the enormous stockpile of tagged photographs we give to Facebook and other social media sites, and the photos governments collect in the process of issuing ID cards and drivers licenses. The technology already exists to automatically identify everyone a camera "sees" in real time. Even without video identification, we can be identified by the unique information continuously broadcasted by the smartphones we carry with us everywhere, or by our laptops or Bluetooth-connected devices. Police have been tracking phones for years, and this practice can now be combined with video analytics.

Once a monitoring system identifies people, their data can be combined with other data, either collected or purchased: from cell phone records, GPS surveillance history, purchasing data, and so on. Social media companies like Facebook have spent years learning about our personalities and beliefs by what we post, comment on, and "like." This is "data inference," and when combined with video it offers a powerful window into people's behaviors and motivations.

Camera resolution is also improving. Gigapixel cameras as so good that they can capture individual faces and identify license places in photos taken miles away. "Wide-area surveillance" cameras can be mounted on airplanes and drones, and can operate continuously. On the ground, cameras can be hidden in street lights and other regular objects. In space, satellite cameras have also dramatically improved.

Data storage has become incredibly cheap, and cloud storage makes it all so easy. Video data can easily be saved for years, allowing computers to conduct all of this surveillance backwards in time.

In democratic countries, such surveillance is marketed as crime prevention -- or counterterrorism. In countries like China, it is blatantly used to suppress political activity and for social control. In all instances, it's being implemented without a lot of public debate by law-enforcement agencies and by corporations in public spaces they control.

This is bad, because ubiquitous surveillance will drastically change our relationship to society. We've never lived in this sort of world, even those of us who have lived through previous totalitarian regimes. The effects will be felt in many different areas. False positives­ -- when the surveillance system gets it wrong­ -- will lead to harassment and worse. Discrimination will become automated. Those who fall outside norms will be marginalized. And most importantly, the inability to live anonymously will have an enormous chilling effect on speech and behavior, which in turn will hobble society's ability to experiment and change. A recent ACLU report discusses these harms in more depth. While it's possible that some of this surveillance is worth the trade-offs, we as society need to deliberately and intelligently make decisions about it.

Some jurisdictions are starting to notice. Last month, San Francisco became the first city to ban facial recognition technology by police and other government agencies. A similar ban is being considered in Somerville, MA, and Oakland, CA. These are exceptions, and limited to the more liberal areas of the country.

We often believe that technological change is inevitable, and that there's nothing we can do to stop it -- or even to steer it. That's simply not true. We're led to believe this because we don't often see it, understand it, or have a say in how or when it is deployed. The problem is that technologies of cameras, resolution, machine learning, and artificial intelligence are complex and specialized.

Laws like what was just passed in San Francisco won't stop the development of these technologies, but they're not intended to. They're intended as pauses, so our policy making can catch up with technology. As a general rule, the US government tends to ignore technologies as they're being developed and deployed, so as not to stifle innovation. But as the rate of technological change increases, so does the unanticipated effects on our lives. Just as we've been surprised by the threats to democracy caused by surveillance capitalism, AI-enabled video surveillance will have similar surprising effects. Maybe a pause in our headlong deployment of these technologies will allow us the time to discuss what kind of society we want to live in, and then enact rules to bring that kind of society about.

This essay previously appeared on Vice Motherboard.

Video Surveillance by Computer

The ACLU's Jay Stanley has just published a fantastic report: "The Dawn of Robot Surveillance" (blog post here) Basically, it lays out a future of ubiquitous video cameras watched by increasingly sophisticated video analytics software, and discusses the potential harms to society.

I'm not going to excerpt a piece, because you really need to read the whole thing.

Rock-Paper-Scissors Robot

How in the world did I not know about this for three years?

Researchers at the University of Tokyo have developed a robot that always wins at rock-paper-scissors. It watches the human player's hand, figures out which finger position the human is about to deploy, and reacts quickly enough to always win.

EDITED TO ADD (6/13): Seems like this is even older -- from 2013.

iOS Shortcut for Recording the Police

"Hey Siri; I'm getting pulled over" can be a shortcut:

Once the shortcut is installed and configured, you just have to say, for example, "Hey Siri, I'm getting pulled over." Then the program pauses music you may be playing, turns down the brightness on the iPhone, and turns on "do not disturb" mode.

It also sends a quick text to a predetermined contact to tell them you've been pulled over, and it starts recording using the iPhone's front-facing camera. Once you've stopped recording, it can text or email the video to a different predetermined contact and save it to Dropbox.

Security and Human Behavior (SHB) 2019

Today is the second day of the twelfth Workshop on Security and Human Behavior, which I am hosting at Harvard University.

SHB is a small, annual, invitational workshop of people studying various aspects of the human side of security, organized each year by Alessandro Acquisti, Ross Anderson, and myself. The 50 or so people in the room include psychologists, economists, computer security researchers, sociologists, political scientists, criminologists, neuroscientists, designers, lawyers, philosophers, anthropologists, business school professors, and a smattering of others. It's not just an interdisciplinary event; most of the people here are individually interdisciplinary.

The goal is to maximize discussion and interaction. We do that by putting everyone on panels, and limiting talks to 7-10 minutes. The rest of the time is left to open discussion. Four hour-and-a-half panels per day over two days equals eight panels; six people per panel means that 48 people get to speak. We also have lunches, dinners, and receptions -- all designed so people from different disciplines talk to each other.

I invariably find this to be the most intellectually stimulating two days of my professional year. It influences my thinking in many different, and sometimes surprising, ways.

This year's program is here. This page lists the participants and includes links to some of their work. As he does every year, Ross Anderson is liveblogging the talks -- remotely, because he was denied a visa earlier this year.

Here are my posts on the first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, and eleventh SHB workshops. Follow those links to find summaries, papers, and occasionally audio recordings of the various workshops. Ross also maintains a good webpage of psychology and security resources.

Chinese Military Wants to Develop Custom OS

Citing security concerns, the Chinese military wants to replace Windows with its own custom operating system:

Thanks to the Snowden, Shadow Brokers, and Vault7 leaks, Beijing officials are well aware of the US' hefty arsenal of hacking tools, available for anything from smart TVs to Linux servers, and from routers to common desktop operating systems, such as Windows and Mac.

Since these leaks have revealed that the US can hack into almost anything, the Chinese government's plan is to adopt a "security by obscurity" approach and run a custom operating system that will make it harder for foreign threat actors -- mainly the US -- to spy on Chinese military operations.

It's unclear exactly how custom this new OS will be. It could be a Linux variant, like North Korea's Red Star OS. Or it could be something completely new. Normally, I would be highly skeptical of a country being able to write and field its own custom operating system, but China is one of the few that is large enough to actually be able to do it. So I'm just moderately skeptical.

EDITED TO ADD (6/12): Russia also wants to develop its own flavor of Linux.

The Cost of Cybercrime

Really interesting paper calculating the worldwide cost of cybercrime:

Abstract: In 2012 we presented the first systematic study of the costs of cybercrime. In this paper,we report what has changed in the seven years since. The period has seen major platform evolution, with the mobile phone replacing the PC and laptop as the consumer terminal of choice, with Android replacing Windows, and with many services moving to the cloud.The use of social networks has become extremely widespread. The executive summary is that about half of all property crime, by volume and by value, is now online. We hypothesised in 2012 that this might be so; it is now established by multiple victimisation studies.Many cybercrime patterns appear to be fairly stable, but there are some interesting changes.Payment fraud, for example, has more than doubled in value but has fallen slightly as a proportion of payment value; the payment system has simply become bigger, and slightly more efficient. Several new cybercrimes are significant enough to mention, including business email compromise and crimes involving cryptocurrencies. The move to the cloud means that system misconfiguration may now be responsible for as many breaches as phishing. Some companies have suffered large losses as a side-effect of denial-of-service worms released by state actors, such as NotPetya; we have to take a view on whether they count as cybercrime.The infrastructure supporting cybercrime, such as botnets, continues to evolve, and specific crimes such as premium-rate phone scams have evolved some interesting variants. The over-all picture is the same as in 2012: traditional offences that are now technically 'computercrimes' such as tax and welfare fraud cost the typical citizen in the low hundreds of Euros/dollars a year; payment frauds and similar offences, where the modus operandi has been completely changed by computers, cost in the tens; while the new computer crimes cost in the tens of cents. Defending against the platforms used to support the latter two types of crime cost citizens in the tens of dollars. Our conclusions remain broadly the same as in 2012:it would be economically rational to spend less in anticipation of cybercrime (on antivirus, firewalls, etc.) and more on response. We are particularly bad at prosecuting criminals who operate infrastructure that other wrongdoers exploit. Given the growing realisation among policymakers that crime hasn't been falling over the past decade, merely moving online, we might reasonably hope for better funded and coordinated law-enforcement action.

Richard Clayton gave a presentation on this yesterday at WEIS. His final slide contained a summary.

  • Payment fraud is up, but credit card sales are up even more -- so we're winning.

  • Cryptocurrencies are enabling new scams, but the bit money is still being list in more traditional investment fraud.

  • Telcom fraud is down, basically because Skype is free.

  • Anti-virus fraud has almost disappeared, but tech support scams are growing very rapidly.

  • The big money is still in tax fraud, welfare fraud, VAT fraud, and so on.

  • We spend more money on cyber defense than we do on the actual losses.

  • Criminals largely act with impunity. They don't believe they will get caught, and mostly that's correct.

Bottom line: the technology has changed a lot since 2012, but the economic considerations remain unchanged.

The Importance of Protecting Cybersecurity Whistleblowers

Interesting essay arguing that we need better legislation to protect cybersecurity whistleblowers.

Congress should act to protect cybersecurity whistleblowers because information security has never been so important, or so challenging. In the wake of a barrage of shocking revelations about data breaches and companies mishandling of customer data, a bipartisan consensus has emerged in support of legislation to give consumers more control over their personal information, require companies to disclose how they collect and use consumer data, and impose penalties for data breaches and misuse of consumer data. The Federal Trade Commission ("FTC") has been held out as the best agency to implement this new regulation. But for any such legislation to be effective, it must protect the courageous whistleblowers who risk their careers to expose data breaches and unauthorized use of consumers' private data.

Whistleblowers strengthen regulatory regimes, and cybersecurity regulation would be no exception. Republican and Democratic leaders from the executive and legislative branches have extolled the virtues of whistleblowers. High-profile cases abound. Recently, Christopher Wylie exposed Cambridge Analytica's misuse of Facebook user data to manipulate voters, including its apparent theft of data from 50 million Facebook users as part of a psychological profiling campaign. Though additional research is needed, the existing empirical data reinforces the consensus that whistleblowers help prevent, detect, and remedy misconduct. Therefore it is reasonable to conclude that protecting and incentivizing whistleblowers could help the government address the many complex challenges facing our nation's information systems.

Fraudulent Academic Papers

The term "fake news" has lost much of its meaning, but it describes a real and dangerous Internet trend. Because it's hard for many people to differentiate a real news site from a fraudulent one, they can be hoodwinked by fictitious news stories pretending to be real. The result is that otherwise reasonable people believe lies.

The trends fostering fake news are more general, though, and we need to start thinking about how it could affect different areas of our lives. In particular, I worry about how it will affect academia. In addition to fake news, I worry about fake research.

An example of this seems to have happened recently in the cryptography field. SIMON is a block cipher designed by the National Security Agency (NSA) and made public in 2013. It's a general design optimized for hardware implementation, with a variety of block sizes and key lengths. Academic cryptanalysts have been trying to break the cipher since then, with some pretty good results, although the NSA's specified parameters are still immune to attack. Last week, a paper appeared on the International Association for Cryptologic Research (IACR) ePrint archive purporting to demonstrate a much more effective break of SIMON, one that would affect actual implementations. The paper was sufficiently weird, the authors sufficiently unknown and the details of the attack sufficiently absent, that the editors took it down a few days later. No harm done in the end.

In recent years, there has been a push to speed up the process of disseminating research results. Instead of the laborious process of academic publication, researchers have turned to faster online publishing processes, preprint servers, and simply posting research results. The IACR ePrint archive is one of those alternatives. This has all sorts of benefits, but one of the casualties is the process of peer review. As flawed as that process is, it does help ensure the accuracy of results. (Of course, bad papers can still make it through the process. We're still dealing with the aftermath of a flawed, and now retracted, Lancet paper linking vaccines with autism.)

Like the news business, academic publishing is subject to abuse. We can only speculate the motivations of the three people who are listed as authors on the SIMON paper, but you can easily imagine better-executed and more nefarious scenarios. In a world of competitive research, one group might publish a fake result to throw other researchers off the trail. It might be a company trying to gain an advantage over a potential competitor, or even a country trying to gain an advantage over another country.

Reverting to a slower and more accurate system isn't the answer; the world is just moving too fast for that. We need to recognize that fictitious research results can now easily be injected into our academic publication system, and tune our skepticism meters accordingly.

This essay previously appeared on Lawfare.com.

First American Financial Corp. Data Records Leak

Krebs on Security is reporting a massive data leak by the real estate title insurance company First American Financial Corp.

"The title insurance agency collects all kinds of documents from both the buyer and seller, including Social Security numbers, drivers licenses, account statements, and even internal corporate documents if you're a small business. You give them all kinds of private information and you expect that to stay private."

Shoval shared a document link he'd been given by First American from a recent transaction, which referenced a record number that was nine digits long and dated April 2019. Modifying the document number in his link by numbers in either direction yielded other peoples' records before or after the same date and time, indicating the document numbers may have been issued sequentially.

The earliest document number available on the site -- 000000075 -- referenced a real estate transaction from 2003. From there, the dates on the documents get closer to real time with each forward increment in the record number.

This is not an uncommon vulnerability: documents without security, just "protected" by a unique serial number that ends up being easily guessable.

Krebs has no evidence that anyone harvested all this data, but that's not the point. The company said this in a statement: "At First American, security, privacy and confidentiality are of the highest priority and we are committed to protecting our customers' information." That's obviously not true; security and privacy are probably pretty low priorities for the company. This is basic stuff, and companies like First America Corp. should be held liable for their poor security practices.

NSA Hawaii

Recently I've heard Edward Snowden talk about his working at the NSA in Hawaii as being "under a pineapple field." CBS News recently ran a segment on that NSA listening post on Oahu.

Not a whole lot of actual information. "We're in office building, in a pineapple field, on Oahu...." And part of it is underground -- we see a tunnel. We didn't get to see any pineapples, though.

Germany Talking about Banning End-to-End Encryption

Der Spiegel is reporting that the German Ministry for Internal Affairs is planning to require all Internet message services to provide plaintext messages on demand, basically outlawing strong end-to-end encryption. Anyone not complying will be blocked, although the article doesn't say how. (Cory Doctorow has previously explained why this would be impossible.)

The article is in German, and I would appreciate additional information from those who can speak the language.

EDITED TO ADD (6/2): Slashdot thread. This seems to be nothing more than political grandstanding: see this post from the Carnegie Endowment for International Peace.

Thangrycat: A Serious Cisco Vulnerability

Summary:

Thangrycat is caused by a series of hardware design flaws within Cisco's Trust Anchor module. First commercially introduced in 2013, Cisco Trust Anchor module (TAm) is a proprietary hardware security module used in a wide range of Cisco products, including enterprise routers, switches and firewalls. TAm is the root of trust that underpins all other Cisco security and trustworthy computing mechanisms in these devices. Thangrycat allows an attacker to make persistent modification to the Trust Anchor module via FPGA bitstream modification, thereby defeating the secure boot process and invalidating Cisco's chain of trust at its root. While the flaws are based in hardware, Thangrycat can be exploited remotely without any need for physical access. Since the flaws reside within the hardware design, it is unlikely that any software security patch will fully resolve the fundamental security vulnerability.

From a news article:

Thrangrycat is awful for two reasons. First, if a hacker exploits this weakness, they can do whatever they want to your routers. Second, the attack can happen remotely ­ it's a software vulnerability. But the fix can only be applied at the hardware level. Like, physical router by physical router. In person. Yeesh.

That said, Thrangrycat only works once you have administrative access to the device. You need a two-step attack in order to get Thrangrycat working. Attack #1 gets you remote administrative access, Attack #2 is Thrangrycat. Attack #2 can't happen without Attack #1. Cisco can protect you from Attack #1 by sending out a software update. If your I.T. people have your systems well secured and are applying updates and patches consistently and you're not a regular target of nation-state actors, you're relatively safe from Attack #1, and therefore, pretty safe from Thrangrycat.

Unfortunately, Attack #1 is a garden variety vulnerability. Many systems don't even have administrative access configured correctly. There's opportunity for Thrangrycat to be exploited.

And from Boing Boing:

Thangrycat relies on attackers being able to run processes as the system's administrator, and Red Balloon, the security firm that disclosed the vulnerability, also revealed a defect that allows attackers to run code as admin.

It's tempting to dismiss the attack on the trusted computing module as a ho-hum flourish: after all, once an attacker has root on your system, all bets are off. But the promise of trusted computing is that computers will be able to detect and undo this kind of compromise, by using a separate, isolated computer to investigate and report on the state of the main system (Huang and Snowden call this an introspection engine). Once this system is compromised, it can be forced to give false reports on the state of the system: for example, it might report that its OS has been successfully updated to patch a vulnerability when really the update has just been thrown away.

As Charlie Warzel and Sarah Jeong discuss in the New York Times, this is an attack that can be executed remotely, but can only be detected by someone physically in the presence of the affected system (and only then after a very careful inspection, and there may still be no way to do anything about it apart from replacing the system or at least the compromised component).

Visiting the NSA

Yesterday, I visited the NSA. It was Cyber Command's birthday, but that's not why I was there. I visited as part of the Berklett Cybersecurity Project, run out of the Berkman Klein Center and funded by the Hewlett Foundation. (BERKman hewLETT -- get it? We have a web page, but it's badly out of date.)

It was a full day of meetings, all unclassified but under the Chatham House Rule. Gen. Nakasone welcomed us and took questions at the start. Various senior officials spoke with us on a variety of topics, but mostly focused on three areas:

  • Russian influence operations, both what the NSA and US Cyber Command did during the 2018 election and what they can do in the future;

  • China and the threats to critical infrastructure from untrusted computer hardware, both the 5G network and more broadly;

  • Machine learning, both how to ensure a ML system is compliant with all laws, and how ML can help with other compliance tasks.

It was all interesting. Those first two topics are ones that I am thinking and writing about, and it was good to hear their perspective. I find that I am much more closely aligned with the NSA about cybersecurity than I am about privacy, which made the meeting much less fraught than it would have been if we were discussing Section 702 of the FISA Amendments Act, Section 215 the USA Freedom Act (up for renewal next year), or any 4th Amendment violations. I don't think we're past those issues by any means, but they make up less of what I am working on.

Fingerprinting iPhones

This clever attack allows someone to uniquely identify a phone when you visit a website, based on data from the accelerometer, gyroscope, and magnetometer sensors.

We have developed a new type of fingerprinting attack, the calibration fingerprinting attack. Our attack uses data gathered from the accelerometer, gyroscope and magnetometer sensors found in smartphones to construct a globally unique fingerprint. Overall, our attack has the following advantages:

  • The attack can be launched by any website you visit or any app you use on a vulnerable device without requiring any explicit confirmation or consent from you.
  • The attack takes less than one second to generate a fingerprint.
  • The attack can generate a globally unique fingerprint for iOS devices.
  • The calibration fingerprint never changes, even after a factory reset.
  • The attack provides an effective means to track you as you browse across the web and move between apps on your phone.

* Following our disclosure, Apple has patched this vulnerability in iOS 12.2.

Research paper.

How Technology and Politics Are Changing Spycraft

Interesting article about how traditional nation-based spycraft is changing. Basically, the Internet makes it increasingly possible to generate a good cover story; cell phone and other electronic surveillance techniques make tracking people easier; and machine learning will make all of this automatic. Meanwhile, Western countries have new laws and norms that put them at a disadvantage over other countries. And finally, much of this has gone corporate.

The Concept of "Return on Data"

This law review article by Noam Kolt, titled "Return on Data," proposes an interesting new way of thinking of privacy law.

Abstract: Consumers routinely supply personal data to technology companies in exchange for services. Yet, the relationship between the utility (U) consumers gain and the data (D) they supply -- "return on data" (ROD) -- remains largely unexplored. Expressed as a ratio, ROD = U / D. While lawmakers strongly advocate protecting consumer privacy, they tend to overlook ROD. Are the benefits of the services enjoyed by consumers, such as social networking and predictive search, commensurate with the value of the data extracted from them? How can consumers compare competing data-for-services deals? Currently, the legal frameworks regulating these transactions, including privacy law, aim primarily to protect personal data. They treat data protection as a standalone issue, distinct from the benefits which consumers receive. This article suggests that privacy concerns should not be viewed in isolation, but as part of ROD. Just as companies can quantify return on investment (ROI) to optimize investment decisions, consumers should be able to assess ROD in order to better spend and invest personal data. Making data-for-services transactions more transparent will enable consumers to evaluate the merits of these deals, negotiate their terms and make more informed decisions. Pivoting from the privacy paradigm to ROD will both incentivize data-driven service providers to offer consumers higher ROD, as well as create opportunities for new market entrants.

Why Are Cryptographers Being Denied Entry into the US?

In March, Adi Shamir -- that's the "S" in RSA -- was denied a US visa to attend the RSA Conference. He's Israeli.

This month, British citizen Ross Anderson couldn't attend an awards ceremony in DC because of visa issues. (You can listen to his recorded acceptance speech.) I've heard of at least one other prominent cryptographer who is in the same boat. Is there some cryptographer blacklist? Is something else going on? A lot of us would like to know.

More Attacks against Computer Automatic Update Systems

Last month, Kaspersky discovered that Asus's live update system was infected with malware, an operation it called Operation Shadowhammer. Now we learn that six other companies were targeted in the same operation.

As we mentioned before, ASUS was not the only company used by the attackers. Studying this case, our experts found other samples that used similar algorithms. As in the ASUS case, the samples were using digitally signed binaries from three other Asian vendors:

  • Electronics Extreme, authors of the zombie survival game called Infestation: Survivor Stories,
  • Innovative Extremist, a company that provides Web and IT infrastructure services but also used to work in game development,
  • Zepetto, the South Korean company that developed the video game Point Blank.

According to our researchers, the attackers either had access to the source code of the victims' projects or they injected malware at the time of project compilation, meaning they were in the networks of those companies. And this reminds us of an attack that we reported on a year ago: the CCleaner incident.

Also, our experts identified three additional victims: another video gaming company, a conglomerate holding company and a pharmaceutical company, all in South Korea. For now we cannot share additional details about those victims, because we are in the process of notifying them about the attack.

Me on supply chain security.

Another Intel Chip Flaw

Remember the Spectre and Meltdown attacks from last year? They were a new class of attacks against complex CPUs, finding subliminal channels in optimization techniques that allow hackers to steal information. Since their discovery, researchers have found additional similar vulnerabilities.

A whole bunch more have just been discovered.

I don't think we're finished yet. A year and a half ago I wrote: "But more are coming, and they'll be worse. 2018 will be the year of microprocessor vulnerabilities, and it's going to be a wild ride." I think more are still coming.

WhatsApp Vulnerability Fixed

WhatsApp fixed a devastating vulnerability that allowed someone to remotely hack a phone by initiating a WhatsApp voice call. The recipient didn't even have to answer the call.

The Israeli cyber-arms manufacturer NSO Group is believed to be behind the exploit, but of course there is no definitive proof.

If you use WhatsApp, update your app immediately.

Cryptanalysis of SIMON-32/64

A weird paper was posted on the Cryptology ePrint Archive (working link is via the Wayback Machine), claiming an attack against the NSA-designed cipher SIMON. You can read some commentary about it here. Basically, the authors claimed an attack so devastating that they would only publish a zero-knowledge proof of their attack. Which they didn't. Nor did they publish anything else of interest, near as I can tell.

The paper has since been deleted from the ePrint Archive, which feels like the correct decision on someone's part.

Another NSA Leaker Identified and Charged

In 2015, the Intercept started publishing "The Drone Papers," based on classified documents leaked by an unknown whistleblower. Today, someone who worked at the NSA, and then at the National Geospatial-Intelligence Agency, was charged with the crime. It is unclear how he was initially identified. It might have been this: "At the agency, prosecutors said, Mr. Hale printed 36 documents from his Top Secret computer."

The article talks about evidence collected after he was identified and searched:

According to the indictment, in August 2014, Mr. Hale's cellphone contact list included information for the reporter, and he possessed two thumb drives. One thumb drive contained a page marked "secret" from a classified document that Mr. Hale had printed in February 2014. Prosecutors said Mr. Hale had tried to delete the document from the thumb drive.

The other thumb drive contained Tor software and the Tails operating system, which were recommended by the reporter's online news outlet in an article published on its website regarding how to anonymously leak documents.

Amazon Is Losing the War on Fraudulent Sellers

Excellent article on fraudulent seller tactics on Amazon.

The most prominent black hat companies for US Amazon sellers offer ways to manipulate Amazon's ranking system to promote products, protect accounts from disciplinary actions, and crush competitors. Sometimes, these black hat companies bribe corporate Amazon employees to leak information from the company's wiki pages and business reports, which they then resell to marketplace sellers for steep prices. One black hat company charges as much as $10,000 a month to help Amazon sellers appear at the top of product search results. Other tactics to promote sellers' products include removing negative reviews from product pages and exploiting technical loopholes on Amazon's site to lift products' overall sales rankings.

[...]

AmzPandora's services ranged from small tasks to more ambitious strategies to rank a product higher using Amazon's algorithm. While it was online, it offered to ping internal contacts at Amazon for $500 to get information about why a seller's account had been suspended, as well as advice on how to appeal the suspension. For $300, the company promised to remove an unspecified number of negative reviews on a listing within three to seven days, which would help increase the overall star rating for a product. For $1.50, the company offered a service to fool the algorithm into believing a product had been added to a shopper's cart or wish list by writing a super URL. And for $1,200, an Amazon seller could purchase a "frequently bought together" spot on another marketplace product's page that would appear for two weeks, which AmzPandora promised would lead to a 10% increase in sales.

This was a good article on this from last year. (My blog post.)

Amazon has a real problem here, primarily because trust in the system is paramount to Amazon's success. As much as they need to crack down on fraudulent sellers, they really want articles like these to not be written.

Slashdot thread. Boing Boing post.

Leaked NSA Hacking Tools

In 2016, a hacker group calling itself the Shadow Brokers released a trove of 2013 NSA hacking tools and related documents. Most people believe it is a front for the Russian government. Since, then the vulnerabilities and tools have been used by both government and criminals, and put the NSA's ability to secure its own cyberweapons seriously into question.

Now we have learned that the Chinese used the tools fourteen months before the Shadow Brokers released them.

Does this mean that both the Chinese and the Russians stole the same set of NSA tools? Did the Russians steal them from the Chinese, who stole them from us? Did it work the other way? I don't think anyone has any idea. But this certainly illustrates how dangerous it is for the NSA -- or US Cyber Command -- to hoard zero-day vulnerabilities.

EDITED TO ADD (5/16): Symantec report.

Malicious MS Office Macro Creator

Evil Clippy is a tool for creating malicious Microsoft Office macros:

At BlackHat Asia we released Evil Clippy, a tool which assists red teamers and security testers in creating malicious MS Office documents. Amongst others, Evil Clippy can hide VBA macros, stomp VBA code (via p-code) and confuse popular macro analysis tools. It runs on Linux, OSX and Windows.

The VBA stomping is the most powerful feature, because it gets around antivirus programs:

VBA stomping abuses a feature which is not officially documented: the undocumented PerformanceCache part of each module stream contains compiled pseudo-code (p-code) for the VBA engine. If the MS Office version specified in the _VBA_PROJECT stream matches the MS Office version of the host program (Word or Excel) then the VBA source code in the module stream is ignored and the p-code is executed instead.

In summary: if we know the version of MS Office of a target system (e.g. Office 2016, 32 bit), we can replace our malicious VBA source code with fake code, while the malicious code will still get executed via p-code. In the meantime, any tool analyzing the VBA source code (such as antivirus) is completely fooled.