Blog Ipsa Loquitur

I enjoyed this piece from Riana Pfefferkorn, writing in NYU Law’s Just Security, on the implications of a forthcoming iPhone feature called “USB Restricted Mode” that blocks all connections to a computer if the phone hasn’t been unlocked in over an hour. Today, even if the police don’t know the passcode to unlock your phone, they can connect your phone to a special computer that will try all possible passcode combinations from 000000 to 999999 over the span of a few days. This fall, Apple will push an update to iPhones that disables that port, and makes it harder to break into a locked iPhone.

As Pfefferkorn explains, the police don’t need a search warrant if there are what’s called “exigent circumstances” – and if the phone is about to lock everyone out of itself forever, Apple might be shooting itself in the foot:

The 2014 Supreme Court case Riley v. California requires police to get a warrant before searching a smartphone seized from someone who’s been arrested. […] “If ‘the police are truly confronted with a “now or never” situation,’ … they may be able to rely on exigent circumstances to search the phone immediately,” the Court said. Id. at 2487 (citation omitted). DOJ could thus thread the needle by arguing that the mere possibility that USB Restricted Mode is enabled on a seized iPhone creates “a ‘now or never’ situation” necessitating the immediate use of Cellebrite or GrayKey without waiting for a warrant.

Applied broadly to iPhones, this exception would swallow the Fourth Amendment’s general rule. The “we need to dump the phone ASAP just in case USB Restricted Mode is on” approach might fly in a one-off situation. But that uncertainty will probably be present most times police seize a locked iPhone. Under that logic, it would be OK to forensically search any iPhone immediately without a warrant, because there would always be exigent circumstances.

That is not how an exception to a rule works. “Exigent circumstances” are supposed to be situational and case-specific. The DOJ’s own manual for electronic evidence search and seizure acknowledges as much: “in electronic device cases, as in all others, the existence of exigent circumstances is tied to the facts of the individual case.” Given that recognition, DOJ would be hard-pressed to adopt or defend a policy allowing warrantless searches of iPhones a priori in all instances on the rationale that every single time police seize an iPhone, they “are truly confronted with a ‘now or never’ situation” as Riley said. That is particularly so given the alternative ways that police, with a warrant, could get data from a locked, encrypted phone, as a recent law review article about Riley explains—for example, going to the relevant service provider and asking for cloud backups of the phone’s contents.

The Fourth Amendment isn’t exactly my strong suit, so this was an extremely educational read.

Published on under This wasn’t covered in ’99 problems’

Sam Rutherford in Gizmodo on how Spain’s Biggest Football App Reportedly Turned Fans Into Unwitting Narcs:

The La Liga app, which is the official streaming app for Spain’s most popular football league, has reportedly been using the microphones on fans’ phones to root out unauthorized broadcasts of matches in public venues like bars and restaurants. It sounds exactly like the kind of surveillance people are afraid of when it comes to modern technology, but as is often the case, the La Liga app technically asks users in Spain for permission to access their mics, according to Spanish Website El Diario.

After downloading the La Liga app, it presents Spanish users with two options: a standard terms-of-service agreement, and a second, opt-in permission that gives La Liga consent to activate your device’s mic and even turn on GPS to help pinpoint the location of unlicensed broadcasts. However, according to the report, the only way you’d know that is by reading the fine print that accompanies the permissions—which no one ever does. Even more troubling, it seems this behavior has been going on for a while, and only recently has been brought back to light thanks to Europe’s new GDPR online privacy laws.

First, it’s nice that modern smartphone platforms have a structured permissions system through which users can grant—and deny—apps access to certain sensors on the phone. The app I use to take notes or write emails doesn’t need access to my GPS coordinates, for example. That was not always the case, and this is definitely an area where smartphone manufacturers have done a good job protecting their consumers.

Second, it makes much more sense for the Googles and Facebooks of the world to offer this kind of surveillance as a service to copyright holders than for companies like La Liga to try to freelance this thing. While El Diario mentions the La Liga app has been downloaded more than 10 million times, that’s still a small fraction of the overall smartphone base.

For example, YouTube scans uploaded videos for copyrighted content, but that might be a result of its tortured history with copyright holders as much as anything else. It’s unclear to me how the La Ligas of the world would convince Google to turn every Android phone into a copyright informant.

Published on under Nosotros Somos El Hermano Grande

Like just about everyone who took an Intro to Psychology course in college, I learned about Philip Zimbardo’s “Stanford Prison Experiment” as a parable about the dangers of groupthink and the ease with which authority—real or imagined—led to the abuse of power. I didn’t know the experiment was fatally flawed until I read The Lifespan of a Lie by Ben Blum:

Zimbardo, a young Stanford psychology professor, built a mock jail in the basement of Jordan Hall and stocked it with nine “prisoners,” and nine “guards,” all male, college-age respondents to a newspaper ad who were assigned their roles at random and paid a generous daily wage to participate. The senior prison “staff” consisted of Zimbardo himself and a handful of his students. The study was supposed to last for two weeks, but after Zimbardo’s girlfriend stopped by six days in and witnessed the conditions in the “Stanford County Jail,” she convinced him to shut it down. Since then, the tale of guards run amok and terrified prisoners breaking down one by one has become world-famous, a cultural touchstone that’s been the subject of books, documentaries, and feature films — even an episode of “Veronica Mars.” ​ The way I learned it, the guard’s behavior was a natural byproduct of the prisoner/guard relationship. However:

Once the simulation got underway, [The “warden,” undergrad student David] Jaffe explicitly corrected guards who weren’t acting tough enough, fostering exactly the pathological behavior that Zimbardo would later claim had arisen organically.

“The guards have to know that every guard is going to be what we call a tough guard,” Jaffe told one such guard. “[H]opefully what will come out of this study is some very serious recommendations for reform… so that we can get on the media and into the press with it, and say ‘Now look at what this is really about.’ … [T]ry and react as you picture the pigs reacting.”

Though most guards gave lackluster performances, some even going out of their way to do small favors for the prisoners, one in particular rose to the challenge: Dave Eshelman, whom experimenters nicknamed “John Wayne” for his Southern accent and inventive cruelty. But Eshelman, who had studied acting throughout high school and college, has always admitted that his accent was just as fake as Korpi’s breakdown. His overarching goal, as he told me in an interview, was simply to help the experiment succeed.

I’m not a sociologist, and I don’t know that a double-blind experiment is the right approach here. (Though it’s fun to imagine a “prison” split in half by bars, and the people on each side are unsure whether they’re prisoners or guards.) But over and over in Blum’s examination of the Stanford Prison Experiment, the researchers seem to have pretty blatantly interfered with the process.

And this isn’t just modern science second-guessing the research methods of yesteryear:

Despite the Stanford prison experiment’s canonical status in intro psych classes around the country today, methodological criticism of it was swift and widespread in the years after it was conducted. Deviating from scientific protocol, Zimbardo and his students had published their first article about the experiment not in an academic journal of psychology but in The New York Times Magazine, sidestepping the usual peer review.

Famed psychologist Erich Fromm, unaware that guards had been explicitly instructed to be “tough,” nonetheless opined that in light of the obvious pressures to abuse, what was most surprising about the experiment was how few guards did. “The authors believe it proves that the situation alone can within a few days transform normal people into abject, submissive individuals or into ruthless sadists,” Fromm wrote. “It seems to me that the experiment proves, if anything, rather the contrary.”

For those of you keeping score at home, the silver lining here appears to be “the situation didn’t turn guards into monsters; some of the guards were monsters this whole time,” which is silver-ish at best. I’ll take it.

Published on under Compliance Will Be Rewarded

Remember the San Bernardino shooters? The husband and wife team that walked into his office holiday party and opened fire with a pair of assault rifles, and were then shot to death by police after a brief car chase? Detectives recovered the husband’s work phone, an iPhone with full-disk encryption enabled.

At the time, the FBI had a whole song and dance routine about how dangerous encryption was, and how companies like Apple needed to build the FBI a system to get keys to unlock any device at any time. But the FBI mishandled the phone in that investigation and made their own jobs a whole lot harder.

As I wrote about it back then:

Look, if local police in a tiny town in the middle of nowhere had screwed up this badly, we’d wonder why the FBI wasn’t entrusted with this. But the FBI made their jobs a lot harder; this can’t be their first time at the iPhone Evidence Rodeo, so how could they have locked themselves out of the phone? […] Sure is funny how the FBI was hilariously inept in their investigation, and now the only way they can get into the phone is by having Apple build a tool to circumvent its own security measures, right?

After all, if you were the FBI, and you wanted a test case, this whole “real live radicalized Muslim terrorist committing an act of war on American soil with pipe bombs and assault rifles before dying in a glorious shootout with the police” thing would be, like, the perfect test case. It sure would be a shame if the traditional investigation went awry, and the only way to get access to the phone was filing a completely novel test case.

That’s a pretty cynical take, I’ll admit. I more or less tiptoed up to the precipice of accusing certain members of the FBI of sabotaging their own investigation; it didn’t seem out of the question that the FBI was using that phone as ammunition in a broader war on encryption. Fortunately, cooler heads prevailed and the only backdoors in my phone are planted by the NSA. Ha ha(?).

Well, earlier this year, the FBI’s Inspector General issued a report on the San Bernardino iPhone investigation. That report is not much kinder to the FBI than my blog post from 2016. Susan Landau, law professor at Tufts University and one of the preeminent experts on this topic, breaks it down for folks like you and I:

For the FBI, the IG report brings some good news: No one deliberately withheld knowledge to prevent opening the locked iPhone. But that’s about the only positive revelation. The IG report chronicles foot dragging during the efforts to open the locked device and, in a critical instance, an aversion to finding a technological resolution of the issue outside of the court case. Above all, the IG report casts doubt on the argument that locked phones are “warrant-proof” devices preventing law enforcement from doing its job.

The FBI’s failure to open the iPhone was a result of bureaucracy and slowdown. Two units of the FBI’s Operational Technology Division (OTD) were key to eventually unlocking the iPhone: the Cryptologic and Electronic Analysis Unit (CEAU), which examines data on digital devices, working largely on criminal cases, and the Remote Operations Unit (ROU), which uses network exploitation techniques and appears to work largely in classified cases. […]

The CEAU and ROU weren’t communicating about this matter, but once they did, things moved pretty quickly:

The ROU chief reached out to his vendors, and on March 16, 2016, discovered that one of them was already 90 percent of the way toward a solution. At the FBI’s request, the vendor reallocated resources, moving work on opening the iPhone “to the ‘front burner.’” A month later, a vendor demonstrated a solution to the FBI, and the court conflict between Apple and the FBI was over.

Opening the locked iPhone should have been a good within the FBI. But that was not the view held by the CEAU chief; he apparently asked the ROU chief, “Why did you do that for?” The CEAU chief told the Inspector General “after the outside vendor came forward, the CEAU chief became frustrated that the case against Apple could no longer go forward.”

That’s a striking story. We have the FBI director testifying—and U.S. attorneys submitting a motion operating of of the same premise—that only Apple could unlock this terrorist’s phone. But it seems that what was really going on, at least on the part of some FBI investigators, was an unwillingness to really try.

Boy, that sure sounds like what privacy experts were publicly worried about back in 2016, doesn’t it?

Published on under Staring Into the Abyss

I really miss the days when there wasn’t a whole month’s worth of news crammed into a week. Last week, the first details of Michael Cohen’s essential(?) consulting business have been leaked into the public. I don’t think there’s anything intrinsically illegal about selling information you have into the mindset of the most powerful person in the world; Cohen’s allowed to shop his thoughts on President Trump to whomever he likes. If your old boss became the President, you could charge AT&T a million dollars to answer their questions about your boss’s favorite model of phone, and what you know about his thoughts on 5G. But it seems some of those shoppers “hired” Cohen awfully soon after Trump tweeted vague threats to upend their industries, which looks a little shady.

More interesting than Cohen’s naked profiteering on the margins is what he probably knows about the heart of the Trump Organization. There were two stories published, by Buzzfeed and USA Today, in January that I think will ultimately result in more indictments than Cohen’s consulting business. The first is by Thomas Frank in Buzzfeed, titled Secret Money: How Trump Made Millions Selling Condos To Unknown Buyers:

In 2008, as the Great Recession cooled real-estate markets, Trump could not make a payment on a bank loan that he had guaranteed personally for $40 million. Trump Entertainment Resorts, which owned the Trump Taj Mahal casino in Atlantic City, faced a $53 million payment to bondholders. Trump forestalled the bank payment by suing the lender, Deutsche Bank. The casino filed for bankruptcy in 2009.

At the same time, Trump became financially entwined with Russians. In March 2008, a Russian billionaire paid Trump $95 million for a Palm Beach, Florida, estate that Trump had bought four years earlier for $41 million. Donald Trump Jr. told a Moscow real-estate conference in June 2008 that his father’s company, the Trump Organization, was planning to build condos and hotels in Russia. And he told a New York conference in September 2008, “We see a lot of money pouring in from Russia.”

Got that? Trump flipped a property for $54m in profit the same year he owed a $53m payment to bondholders, which is also the same year as the Don Jr. quote above. One does wonder.

But it’s not just the fact that there was “money pouring in.” It’s that shell companies hide the identifies of the buyers. Anyone can set up a company like this in an afternoon, and there are no laws requiring transparency into the real owners of the shell company. And there are tens of millions of dollars of transactions with shell companies during the years when Trump Jr. liked to brag about Russian money. Frank continues:

Trump Jr. was executive vice president of development and acquisitions at the Trump Organization, which opened two major condo towers in early 2008 after a four-year lull. By the time Trump Jr. made his now-famous comment in September 2008, cash-paying shell companies had bought $43 million worth of condos at the Trump International Hotel and Tower Chicago and at the Trump International Hotel Las Vegas.

At a Trump-licensed condo building in Miami-Dade, cash-paying shell companies had bought $32 million worth of condos.

During this time, the future president and his children also were heavily promoting the Trump SoHo, a lower Manhattan high-rise that has been mired in controversy. In his September 2008 remarks, Donald Jr. cited the project: “In terms of high-end product influx into the US, Russians make up a pretty disproportionate cross-section of a lot of our assets; say in Dubai, and certainly with our project in SoHo and anywhere in New York.”

And Nick Penzenstadler in USA Today reported earlier this year that most folks buying real estate from the President’s company are using shell companies to hide their identity:

The trend toward Trump’s real estate buyers buyers obscuring their identities began around the time he won the Republican nomination, midway through 2016, according to USA TODAY’s analysis of every domestic real estate sale by one of his companies.

In the two years before the nomination, 4% of Trump buyers utilized the tactic. In the year after, the rate skyrocketed to about 70%. USA TODAY’s tracking of sales shows the trend held firm through Trump’s first year in office.

Profits from sales of those properties flow through a trust run by Trump’s sons. The president is the sole beneficiary of the trust and he can withdraw cash at any time.

I understand some of the uproar about the not-hush-money portions of Cohen’s “consulting” business, but if I were looking to launder money or just bribe the President, I wouldn’t funnel the money through Cohen to do it. It seems much easier to set up a shell company and simply buy a $2 million condo for $10 million.

Published on under Maybe Let's Not Make a Deal, Okay

Maciej Cegłowski is one of the best writers about the internet you can read. In April 2017, he gave a talk titled Build a Better Monster: Morality, Machine Learning, and Mass Surveillance that you should watch or read in its entirety. At base, the talk is about Surveillance Capitalism, which is the economic basis of the Internet. As Cegłowski puts it, “every interaction with a computing device leaves a data trail, and whole industries exist to consume this data.”

Here’s his bit about the advertising industry:

Ads are served indirectly, based on real-time auctions conducted when the page is served by a maze of intermediaries. This highly automated market is a magnet for fraud, so much of the complexity of modern ad technology consists of additional (and invasive) tracking.

Curiously, despite years of improvements in the technology, and the amount of user data available to the ad networks, online advertising isn’t targeted all that well. You can convince yourself of this by turning off your ad blocker for a week. In a recent example, Chase stopped serving ads to 95% of its websites and saw no measurable difference in ‘engagement’ metrics.

Many advertisers are simply not equipped to use the full panoply of surveillance options. More importantly, adversaries have become very good at gaming real-time ad marketplaces, which introduces noise into the system. An uncharitable but accurate description of online advertising in 2017 is “robots serving ads to robots”. A considerable fraction (only Google and Facebook have the numbers) of the money sloshing around goes to scammers.

So robots bid against one another for the right to show ads on pages, and other robots visit pages with ads to drive up the value of the pages with ads on them in the first place. Of all of humanity’s creations, this quasi-ecosystem has to be one of the most baffling.

As an aside, even the biggest and ostensibly best surveillance companies still haven’t gotten the hang of this stuff. Facebook recently showed me that three of my friends had recently visited New York City, and encouraged me to visit New York City as well. Somehow, Facebook’s system failed to account for the fact that all three of those friends—not to mention myself—live in New York City.

That’s not to say that this demonstrates Facebook is somehow lousy at surveillance. This is just a funny outlier in the midst of surveillance so scary-good that it’s hard to say with certainty that Facebook isn’t listening to the conversations you have in front of your phone. Heck, IBM’s Watson answered a Jeopardy question in the “U.S. Cities” category with “Toronto” en route to crushing its human competitors. The more capable these systems get, the funnier the outliers.

But Also

The outliers serve a second purpose, according to Cegłowski. This is one of his best arguments:

The relative ineffectiveness of targeted advertising creates pressure to collect more data. Ad networks are not just evaluated by their current ad revenue, but by expectations about what new ad formats will make possible in the future, in a dynamic I’ve called “investor storytime”. The more poorly current ads perform, the more room there is to tell convincing stories about future advertising technology, which of course will require new forms of surveillance.

This trick of constantly selling the next version of the ad economy works because new ad formats really do have better engagement. Advertising is like a disease: it takes people time to develop immunity and resistance. Even the first banner ad had a 70% click through rate.

So long as advertising is the economic engine of the internet, the march toward ever more invasive surveillance technologies and ever creepier ads is inexorable. ​Toward that end, Cegłowski shares some meditations on what might make the ads of the future creepy in a way that’s hard to really wrap your head around. Advertising will be powered by artificial intelligences, but AIs are inherently alien, mostly because we don’t understand enough about brains to be able to reinvent them.

In the past, we assumed that when machines reached near-human performance in tasks like image recognition, it would be thanks to fundamental breakthroughs into the nature of cognition. We would be able to lift the lid on the human mind and see all the little gears turning.

What’s happened instead is odd. We found a way to get terrific results by combining fairly simple math with enormous data sets. But this discovery did not advance our understanding. The mathematical techniques used in machine learning don’t have a complex, intelligible internal structure we can reason about. Like our brains, they are a wild, interconnected tangle.

The result is that the algorithms that decide what we see (ads and contents) are smarter than us in some ways, and dangerously unfit to decide how to filter the word for us in other ways. The future’s going to be weird!

Published on under Gazing Into the Abyss