On February 4, Google proudly announced that its Nest Secure home anti-intrusion device would now support its voice-activated Google Assistant service. There was a catch, however: No one knew that the Nest Secure actually had a microphone inside it. Google claims the microphone “was never intended to be a secret” and “has never been turned on,” but it’s hard to shake the feeling that hidden mics are a natural step for Silicon Valley giants intent on collecting as much data as possible, no matter the cost to user privacy.
This episode is a perfect encapsulation of the digital threat outlined in a new book by tech critic and Harvard Business School professor Shoshana Zuboff, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power. First published as an essay in a German publication in 2014, what Zuboff describes amounts to a new economic logic hatched in corporate America that aims to extract staggering value from users’ private lives. Cataloguing a dizzying array of sensors and invasive software, Zuboff sketches a vision of the economic future in which companies race to collect data in pursuit of Facebook- or Google-like profits.
Zuboff, who published her first book in 1989 on the future of technology and data in the workplace, warns in Surveillance Capitalism of a “seventh extinction” that threatens to eradicate “what has been held most precious in human nature.” Given the fragility of the global political and economic order, surveillance capitalism amounts to a “coup from above,” Zuboff argues, an assault on democracy by way of subverting the very idea of what it means to be an individual.
I recently spoke with Zuboff about her book, what constitutes surveillance capitalism, how it operates, and what sets it apart from other historical changes in the economy. The following interview has been edited and condensed for clarity and length.
I’m probably just gonna hold the recorder close to you because there’s a lot of ambient noise in here. The first thing I wanted to ask was for a definition of surveillance capitalism for the uninitiated. How would you describe it?
Surveillance capitalism follows this pattern but with what I would call a kind of dark twist. And that is it claims private human experience as a source of free raw material, subordinated to the market dynamic and reborn as behavioral data. Those behavioral data are then combined with advanced computational capabilities in order to produce predictions of human behavior. So all of these behavioral data now streamed into our 21st-century dark Satanic mills, which are what we call machine intelligence, machine learning, artificial intelligence, in order to spew out predictions. These are a new kind of product.
There’s data and information collected about people that is used specifically to inform the particular products that are served back to them, but there’s this other data, what you’ve termed behavioral surplus, that’s information that doesn’t have an immediate use but is itself a kind of control and power that these organizations possess that gives them an advantage over other companies. Why is behavioral surplus something that’s so critical in surveillance capitalism?
So the point is that there’s a meta level of these data that have tremendous predictive value that you don’t know you’re communicating when you are posting or when you’re searching or when you’re browsing or all of these things, and that’s data that is more than was needed for product and service improvement. These extra data initially, at the beginning of all of this, were lying around, unused, considered data exhaust, waste material. Eventually it was discovered that they had significant predictive value and that’s what was used to create the kind of prediction products of where people would click that became the basis for these new online advertising markets. So the idea here is that there is behavioral data that companies are collecting about us, that are being used to improve what they give us, but there is much more behavioral information that we are communicating that we don’t know. And this is the surplus that they then take for its predictive value, stream it through their production processes to create these prediction products, and all of this is happening without our permission, without our knowledge. It’s happening in a way that is designed to bypass our awareness, it’s happening in a way that is engineered to keep us ignorant.
This is what I call the shadow text. It’s what they can lift from these behavioral flows that gives them tremendous predictive power, and it’s different from what we knew that we were giving them. Which is why today under some of the new regulations, for example GDPR, it says, well, you can go to a company and you can ask them for the data they have on you. But when you go to a company and ask them for the data they have on you, what you’re really talking about is the data that you’ve already given them. But the important data they have on you is this metadata, this stuff that they’ve been able to pull from your data that you don’t even know about. Like if you use exclamation points or if you said 6:45 or later, if you use bullet points instead of just a general paragraph. And a million other things. So it’s this surplus that is more than what is needed for products and service improvement that became critical to the fabrication of these prediction products and laid the basis for this new work.
Just to pick an isolated strand of that data, how does whether or not I prefer to use periods or ellipses or semicolons become something that has predictive power that’s valuable?
You could then theoretically arrange that data in such a way and get something that’s useful to whatever the possessor of that data’s purpose is or agenda is.
Why is surveillance capitalism such a departure from just capitalism before? It’s something you address in the book to some extent, and how it departs from managerial capitalism and other traditions of capitalism but I’m kind of curious about what’s the value of having particular nomenclature to describe surveillance capitalists?
So even early on in the theorizing of capitalism it was understood that capitalism takes on different market forms and different eras in the context of different technologies. We’ve had mercantile capitalism, and we’ve had factory capitalism, and mass-production capitalism, and managerial capitalism, financial capitalism. And typically what happens in these new concepts is that modifier, like “mass production,” or in my case “surveillance” capitalism, what that modifier is doing is pinpointing the pivot of value creation in this new market form. So, for example, mass-production capitalism: What’s creating the value in this new form of capitalism is the idea that you have a minute division of labor and standardization of parts. The separation of execution and administration, the high volume, low-unit cost which was the outcome, all depended upon having these particular characteristics of mass production that produced the value that ended up resulting in a new economic logic that yielded high volume and low-unit cost.
One of the things that big Silicon Valley companies arguably have in common, more than how they’re able to make use of consumer data, is that they’re able to construct software and hardware platforms to which entire economies have had to accommodate themselves. They create their own sort of protected gardens in which they can operate and in which they can discriminate. I’m curious about the ways in which the nature of the platform is actually what informs the economic power and ascendance in the structure of this new form of capitalism.
What’s an example of a platform without surveillance capitalism? Are there any current examples that come to mind for you?
This surveillance capitalism is an economic logic now that is travelling not only across the tech sector, but traveling across sectors in the regular economy where platforms are not the prominent constellation. So in the insurance industry, in the health-care industry, insurance companies are using telematics so they know how you’re driving in real time, and can reward and punish you with higher and lower premiums in real time for whether or not your driving costs them more or less money, or whether or not your eating costs them more or less money, or whether or not your exercise patterns cost them more or less money. Health-care providers who are using telematics for not only feeding into these prediction markets but also collecting all kinds of ancillary data to sell to third parties, and being part of these whole ecosystems now, are behavioral surplus suppliers. You download a diabetes app, it takes your phone, it takes your microphone, it takes your camera, it takes your contacts. Maybe it helps you manage your diabetes a little bit, but it’s also just a part of this whole supply-chain dynamic for behavioral surplus flows.
And this is an economic logic that is only now made possible because of the technological developments of the last 20 years.
Fast-forward 20 years and now those simple closed loops are things like the Nest thermostat, where scholarly analysis indicates that if you are a vigilant consumer and you install a Nest thermostat in your bedroom, you should review a minimum of 1,000 privacy contracts, because everything is flowing to third parties which is flowing to third parties, which is flowing to third parties. No one takes accountability for the third parties, and by the way, if you don’t agree to each policy at each stage, then you begin to lose the functionality of the product and the reason that you bought it in the first place.
In your first book, In the Age of the Smart Machine, you outline two distinct paths for how the future might unfold, and the ways technology might evolve. What do you see as having changed since that time — between 1989 and the 2000s — that led to this situation?
That was a story about how, if we don’t pay attention to the opportunities in what I called an information workplace, where, you know, people need to get educated to be included in this kind of work, and it’s a more intellectual work, and it’s less hierarchical and everyone can make contributions — if we don’t pay attention, it’s just going to be top-down information, and it’s going to be used for control and surveillance, and that’s what I call the information panopticon, which was the end of that book.
And I talked about either going this way or that way, there’s neoliberalism and re-engineering and shareholder value maximization, and no one really giving a toss about educating the workforce and really taking advantage of these new technologies. It was all about offshoring and outsourcing and cost-cutting and down, down, down, down. So, of course it all went to the surveillance things, so that now workplaces have really become the laboratories for surveillance capitalism because that’s where we have habituated human beings to perpetual surveillance.
Amazon’s a good example of a pioneer in that, to your point earlier. They may have been late to it with consumers but they were absolutely pioneers of it in terms of their own workforce.
Whatever darkness I foresaw there has come through and much, much more. But now, the thing is, we’re in the 21st century, and so now these same capabilities, this textualization of the world, the idea that everything is now information — well, this [has] spilled over the walls of the factories and the offices and the workplaces. If those struggles were between capital and labor, the struggles today are between capital and society. Now this stuff is bearing down on all of us because we’re the users, you know. It’s not about our roles, our economic roles as workers, as employees — now our interface with these forces of capital is just as regular people in the course of our daily lives. We’re called users.
So our app user is like the new prole — I don’t want to be glib, but it seems like what you’re describing is a major reconfiguration —
So the capitalists came for the trees, then they came for the labor, and now they’re coming for individual, private lives. Do you have any idea what could possibly come next?
A counterpoint: These companies are really bad at predicting what their consumers will do in the future. One of the biggest places Amazon’s advertising dollars go is into retargeting — ads showing you what you’ve already looked at. And they believe that showing you what you’ve already looked at is a good way to get you to buy more of that thing. And that’s where they get the most value now, it’s not a predictive thing, it’s just literally regurgitating what you’ve already bought back at you. Are these companies on the precipice of developing far more sophisticated predictive powers that can make use of the data they’re collecting?
So this is the telematics, this is the stuff that, you know, we can not only know how you’re driving and where you’re driving, we can know the gaze of your eyes — and that’s really important for insurers, to know if you’re driving safely. And we can know what you’re talking about in your car, and, like, Amazon and Google and so forth are already in a contest for the car dashboard, because that’s a way of hearing what you’re talking about and knowing where you’re going. So they talk about, you know, shopping from the driving wheel. So now the automobile itself becomes this little surveillance bubble. We can get all of this information, from your conversation to your shopping to where you’re going to what you’re doing to how you’re driving. And this has predictive value for all kinds of business customers.
It also seems to me, though, to be contingent on the capacity of these companies to make technological developments. How do we know that the surveillance capitalist is going to actually have power going forward? It’s scary, and they’re doing a lot of excessive data collection already that violates any conceivable norm, but it also seems like these companies are struggling to develop the power to get even the data that they claim to want in the future.
What you’re seeing is a shift to this idea of, it’s not a product, it’s a smart product, it’s not a service, it’s a personalized service. This is the shift to surveillance capitalism, it’s a shift to this sort of parasitic form that instead of, “We’re putting all our effort into making the best product for you,” or the best service for you, and really solving the problems in your life and really making your health better, helping your financial situation and creating more employment opportunities, you know, it’s this …
I guess part of what I find curious is that some businesses are really good at this and others are really bad. You use these apps and they’re all terrible. They’re not great. Do you think that these laggards in surveillance capitalism pose any threat to the viability of that economic logic?
Mass production began as a very violent form. And there was no law to constrain it or impede it, and, there were unsafe working conditions and people were paid slave wages and children were working in factories. And it took decades and public contest and mustering democratic resources and eventually law and regulation that shaped and tamed it and made it something that was actually palatable for society in a relative equilibrium that we call market democracy. And I believe that surveillance capitalism has gone very far in some sectors, and is moving across other sectors. That’s why Ford to me is such an iconic example: This is what businesses are looking to, here’s where the margins are gonna come from, this is gonna be the new solution for how we make profit in the information age. The dominant form of capitalism is actually something where the main thing that we sell and buy are behavioral futures.
So this is why it’s so critical for us to understand this, because we’re at the beginning of this art, not the end of it. This has developed over the last 20 years with barely any impediment from law, with barely any impediment from regulation. It has developed while democracy has slept. And I give — I explore 16 reasons, in-depth, including some historical reasons why that is the case, why it has had basically a 20-year free run to develop.
But my argument is, this is why now, you know, the time has come where enough of this is substantiated out there that we can understand it, that we can see how it works. We can see its aims and its goals and its results. When I write about the Facebook contagion experiments and Pokémon Go, I write about these as sort of population-scale experiments in how to do surveillance capitalism in the real world, at scale. So Pokémon Go is a dry-run for the Google City. Right? Google City is a population-level experiment in how you herd and tune populations, modify people’s behavior, to steer them toward guaranteed commercial outcomes, so that the game is really based on Pokémon Go’s Niantic Labs which, of course, is a Google-incubated operation. It’s hosting its own behavioral futures markets, it has business establishments, McDonald’s, and so forth, paying it for footfall in exactly the same way that online advertisers pay for click-through rates.
More than Amazon marketplace customers pay to play on that platform —
So this is the template for the Google City, the smart city, where the whole digital architecture becomes a global means of behavioral modification that we use to tune and herd populations toward guaranteed commercial outcomes in the city. And this is why citizens in Toronto now are contesting the idea of Google having the waterfront, because the whole idea is now these computational systems replace politics, and the whole idea is that they’re gonna gently, with a smile, tune and herd people toward the outcomes that serve Google’s commercial goals and the commercial goals of its business customers in these prediction markets.
It sounds almost like it’s a means of governance, as an algorithm.
I feel like I’d be remiss if I didn’t bring up China. To what extent do you think China offers a model of what the future may look like or how it’s distinct from it?
No, they just want our money.
I call this instrumentarian power, in contrast to totalitarian power. Instrumentarian power. Two reasons: One, it relies on the instrumentation of the digital milieu because it’s through that medium that we are being tuned and herded and shunted and coached and modified. No one’s coming up to like, you know, slap you or kill you or hurt you …
It sounds kind of totalitarian, in that people are just stripped of agency.
That’s our pivot. Now look at China. What is China? China is an authoritarian state, not a democratic state. So, authoritarian state. Inside China, these internet companies which amassed also enormous, enormous instrumentarian capabilities, enormous asymmetries of behavioral knowledge, predictive power, analytical capability, able to reward and punish behavior in these very finely grained ways. Buy this, discount that, so and so forth. An authoritarian state sees these instrumentarian capabilities and says, “This is perfect. We want to take these, not terror, not murder, not the gulag. We wanna take these instrumentarian capabilities and now pivot them to the political and social outcomes that we as an authoritarian government seek. This is how we want our population to behave and this is how we’re gonna discipline our population. And these are the parameters we’re gonna give them. And this is how we’re gonna restore order in a society that has fallen into chaos because of the complete destruction of social trust in the wake of decades of the Maoist project.” So, what you see in China is the marriage of authoritarian state with instrumentarian power. And that is a very dark and dangerous endgame.
You describe it as an endgame, and I don’t want to conflate things, but having earlier described the moment we’re at as an initial stage of surveillance capitalism —
And China represents one place where the dynamic could change and where —
So, this is what surveillance capitalism represents. And this where my book ends up, and the final chapters represent a profound threat to democracy that goes beyond the brief of just another capitalism that makes a lot of money. It increases mass consumption, and employs a lot of people, and gives consumers what they want. Now we’re talking about a form of capitalism that in order to fulfill itself, its own imperatives of scale, scope, and action, its own imperative of prediction, in order to fulfill its own imperatives, ultimately it replaces society and replaces politics with these computational principles, which right now are aimed toward commercial outcomes.
But those principles could very easily shift.
The way this can be repurposed for political ends, not commercial ends.
And so, what is under siege here is democracy, from two directions. The structure created under the auspices of private capital that is based on unprecedented asymmetries of knowledge — knowledge that is about us but not for us, that gives rise to unprecedented asymmetries of power, a power that is able to shape our behavior but that operates outside of our awareness and is designed to operate, always keeping us ignorant. Engineered to keep us ignorant. We’re entering the 21st century in this institutional setting that introduces a whole new axis of social inequality. Not just economic inequality now, but these profound inequalities of knowledge and the power that accrues to behavioral knowledge that can actually influence our behavior, influence the behavior of our group, of our city, of our region, of our country, of our society. That’s at the institutional level and that is disfiguring 21st-century society, right from the beginning.
At the individual level, we are the so-called users whose behavior is being intervened upon, touched, coaxed, modified, contaged, influenced in all of these ways that are designed to be undetectable to us, to bypass our awareness. So, this is an intervention at the level of human agency, at the level of human autonomy, at the level of individual sovereignty, at the level where we expect to have an idea about our future that we act on now — I’m gonna meet Noah, so I get up early in the morning and I do a bunch of things that are gonna get me to the place where eventually I’m gonna meet Noah, because that’s what I choose to do. That’s my action. That’s what we call free will, that’s what we call agency, and that’s what we call moral autonomy. We can’t imagine a democratic society without human beings who can act this way. It was part and parcel of the conception of a self-regulating demos from the beginning.
Now we’re entering the 21st century with these threats to democratic society coming both from the structural level and from the individual level, the level of our intimate behavior. And when you put these two things together, that becomes something that’s, you know, bigger than what we normally associated with capitalism. As we said at the beginning of our conversation, Noah, that [was a] capitalism that bore down on the economic domain and it was in our workplaces and in our economic roles. Now, this thing, because of its drive for totality, for total information, for total certainty, for scale, for scope, for action, these imperatives that drive it … It’s into society, it’s into every aspect. When Eric Schmidt heard that the Toronto public officials had given Google the go-ahead to take the part of the waterfront to turn it into this smart city thing, I might be slightly paraphrasing, but Eric Schmidt’s words were, “Oh great, now it’s our turn.”
I remember this.
Surveillance capitalism really, as we said before, it moves beyond the economic domain, to the societal domain, into our lives, threatening the very basis of a democratic social system. And that’s where, to me, we now recognize that the time has come for us to name it, to understand it, to recognize that it’s an economic logic that is not a necessary consequence of the digital or of the platform mechanism and that this is the time when we now, through understanding and through naming, we change our consciousness, we turn to our democratic institutions and reclaim democracy as a source of the law and the new regulatory institutions that are gonna intervene upon, interrupt, and outlaw these mechanisms, so that the age of surveillance capitalism turns out to be a short age. This is not legitimate. And the choices with which it has left us, as 21st-century citizens, are not legitimate choices. We should not be having to choose between having our experience scraped for other’s purposes and the basic requirements of effective social participation.
What do you think of the emerging political currents aligned against these kinds of companies pursuing this economic logic?
It also struck me that these were people who were popular in the public imagination because they were paying people really well, and they were offering services that were free, and so on. There was a cultural cachet that’s now cracking. What do you make of the efforts to build political agendas meant to further destabilize it?
Ultimately, I see it as a political discussion that’s gotta be driven by a sea change in public opinion. Because everywhere I go, people are fed up with this, but they haven’t known how to name it. It’s a sense of unease, of anxiety, of loss of control, a sense of being manipulated, a sense of loss of freedom. A sense of powers that we don’t understand. And as they are able to name it, the sense of its intolerability becomes truly palpable. These shifts in public opinion are what ultimately are gonna force elected officials to pay attention. This is going to be a new era of law, a new era of regulation, and it’s got to be designed for this 21st-century situation, where we have unprecedented economic mechanisms that are going to require unprecedented legal remedies.
At the same time, I also believe that once we begin to double down on democracy and democracy sort of awakens, the sleeping giant of democracy awakens to actually confront this stuff, we’re creating space for new competitive solutions. Because we need different kinds of companies, and different kinds of capitalists, and different kinds of ecosystems and alliances that are actually capable of reclaiming the digital for the kind of values and functionality that we wanted from it in the first place. Every single piece of research, going all the way back to the early 2000s, shows that whenever you expose people to what’s really going on behind the scenes with surveillance capitalism, they don’t want anything to do [with] it. The only reason we keep engaging with it is because we feel like we have no choice. So, as soon as there is real choice for people that can also provide effective action, these new competitors have the opportunity to have literally every person on Earth as their customer. I mean, that’s really what it’s about right now.
Another way to look at surveillance capitalism, from the market point of view, is as a colossal market failure. Because it is not giving people what people want. It’s giving business customers what they want, to be able to manipulate people, but it’s not giving actual populations of people what we want. So, it has severed the traditional reciprocities between capitalism and its societies, where it counts on its society as a source of customers and a source of employees, and reoriented itself toward these prediction markets of business customers. But in the process, you know, our problems are not being solved. Our health-care problems are not being solved. Our climate problems are not being solved. Our employment problems are not being solved. How to build a better car, how to build a better building, how to make a better city that is democratic and still contains pollution and all those other things. These problems are not getting solved, surveillance capitalism is scraping off data for other purposes that are not solving our core issues.
The trade-off that we’re sort of baked into believing is that, “Well, the price I pay is that I’m giving up a bunch of shit I don’t know about because I can watch The Simpsons whenever I want with maximum convenience.” We’ve sort of implicitly accepted that there’s a trade-off already. We didn’t even have to be duped, we didn’t even have to be presented with a choice because it was made for us and we seemed okay with it. I mean, that’s the image that’s presented. I don’t quite buy that.
We download a diabetes app. Just downloading it allows x percentage of the diabetes apps to take all the contacts on your phone, x percentage take the contacts and the mic, x percentage takes the contacts, the mic, and the camera. The stuff that they’re taking from you has nothing to do with the diabetes functionality for which you downloaded the app. Absolutely nothing. It’s simply siphoning off data to third parties for other revenue streams that are part of these surveillance capitalists’ ecosystems. This is a hidden world that is constructed around us in these elaborate supply chains. Everything that we’re touching that is internet-enabled is essentially a supply chain interface, but it is all designed to be hidden.
This is a conversation that hasn’t happened yet. And if there’s a reason why people are really digging this book I’ve written [for which] I’ve spent so long trying to discern these mechanisms, name them, and bring them forth so that people can have them to think about and talk about, it’s that all of a sudden, here’s this thing I’ve been feeling, this problem I’ve been feeling that I didn’t really know what it was and I didn’t have a name for it, and now I can name it. And when I can name it, I can think about it, and there’s a language. And once you have that, really, the landscape changes and the power dynamics change. And I don’t mean just my book. There are many other inputs to this process. But as we begin to name, this whole power dynamic is gonna change, because this stuff can no longer get away with being secret, it can no longer get away with being surveillance. And when the jig is up on that, then we’re cutting at the core of its economic umbilical cord.
- Beginning of New Era of Democracy in Pakistan
- Corporate America & Wall Street - The Latest Terrorist Threat! (Part 1)
- White Domestic Terrorism as a National Security Threat Presents Challenges for 2020 Presidential Candidates
- What You Don't Know is Fooling You, Indeed! Which Islam Are You Talking About? Part III of IV
- Ezine Ads: the Spam Filters Threat
- Business and Home Security - The Role of CCTV
- The curse of technology
- Investor Presentation
- A Challenge to the Moral Reputation of a Nation
- 9/11 And Lebanon - Were They Templates For "On-the-March" Islam Taking Over Western Civilization?
- Skulking Reapers
- The Final Perspective
- Victory in Iraq!
- Burnt Generation of Iran
- U.S. House passes legislation taking hard line on China over Hong Kong, Huawei
- 'He's the most corrupt president in all our history!' Joe Biden tears into Donald Trump as 12 2020 Democrats take to the stage for primary debate - with field united in slamming president and backing impeachment
- China and the Global Recession - Part 1 - The Domestic Situation
- Revive Lincoln's Monetary Policy - An Open Letter to President Obama
- Rethinking the Burqa in the Age of Obsession
- Ultralight Backpacking Water Treatment
Shoshana Zuboff Talks Surveillance Capitalism’s Threat to Democracy have 5493 words, post on nymag.com at February 24, 2019. This is cached page on Europe Breaking News. If you want remove this page, please contact us.