The Philosophy of Online Manipulation

Front matter

Ul

The Philosophy of Online Manipulation

Are we being manipulated online? If so, is being manipulated by online technologies and algorithmic systems notably different from human forms of manipulation? And what is under threat exactly when people are manipulated online?

PART I Conceptual and methodological questions

Online manipulation: Charting the field FLEUR JONGEPIER AND MICHAEL KLENK

How philosophy might contribute to the practical ethics of online manipulation ANNE BARNHILL

Online manipulation and agential risk MASSIMILIANO L. CAPPUCCIO, CONSTANTINE SANDIS, AND AUSTIN WYATT

Manipulative machines JESSICA PEPP, RACHEL STERKEN, MATTHEW MCKEEVER, AND ELIOT MICHAELSON

Manipulation, injustice, and technology MICHAEL KLENK vill

Contents

PART II Threats to autonomy, freedom, and meaning in life

Commercial Online Choice Architecture: When Roads Are Paved With Bad Intentions THOMAS NYS AND BART ENGELEN

Microtargeting people as a mere means FLEUR JONGEPIER AND JAN WILLEM WIELAND

Manipulation as digital invasion: A neo-republican approach MARIANNA CAPASSO

Gamification, Manipulation, and Domination MOTI GORIN

Manipulative Design Through Gamification W. JARED PARMER

Technological Manipulation and Threats to Meaning in Life SVEN NYHOLM

Digital Manipulation and Mental Integrity GEOFF KEELING AND CHRISTOPHER BURR

PART M Epistemic, affective, and political harms and risks

Is There a Duty to Disclose Epistemic Risk?

HANNA KIRI GUNN

Promoting Vices: Designing the Web for Manipulation LUKAS SCHWENGERER

Online affective manipulation NATHAN WILDMAN, NATASCHA RIETDIJK, AND ALFRED ARCHER

Manipulation and the Affective Realm of Social Media ALEXANDER FISCHER

Social media, emergent manipulation, and political legitimacy ADAM PHAM, ALAN RUBEL, AND CLINTON CASTRO

Contents ix

PART IV Legal and regulatory perspectives syal 19 Regulating online defaults 373

20 Manipulation, Real-Time Profiling, and their Wrongs 392 JIAHONG CHEN AND LUCAS MIOTTO

Index 410

Acknowledgements

The idea for this volume was born pre-pandemic during Michael’s Niels Stensen Fellowship on the ethics of social media, which focused on questions about manipulation. This being before the pandemic, an international workshop was planned, with the usual invitations, trans-continental flights, paper presentations, and a dinner in mind. Things turned out differently, and we had to change tack. With a physical workshop out of sight, and budget for the workshop freed up, a plan for an open access volume began to emerge. Fleur worked on related topics during her VENI project on algorithms and autonomy, and we decided to team up. With Fleur aboard, we set out to plan the volume open access and, in addition, to take a risk and set up an online workshop series dedicated to the discussion, sharing, and improving early drafts of the individual contributions of this volume and the book as a whole. Together with our contributors and the (regular) participants of the workshop series, we got a chance not only to discuss new philosophical approaches, distinctions, and conceptual tools but also to experiment with creating and engaging in an online research community. It has been an intense and illuminating experience, and we hope that our contributors and participants found it as rewarding and stimulating as we did. What could have been “just” another philosophy workshop, fun and illuminating for a few, turned into an accessible workshop series of ten sessions, attended by a considerable number of people, and ultimately led to this book, available for everyone. It was an opportunity afforded by the changed circumstances of the pandemic. It took more time to organise, and we missed many of the more free-flowing and rewarding interactions of inperson meetings. So, at least as online philosophy is concerned, we come down with a measured judgement: the online world offers great opportunities for making philosophy accessible and open to everyone across the globe, yet we were also powerfully reminded of the value of real-life interactions and hope to meet many of our contributors and participants in the offline world in the future.

We are immensely grateful to the Niels Stensen Fellowship and the Dutch Organisation for Scientific Research (NWO) that made the open access

Acknowledgements xi publication of this volume possible. We hereby also want to thank everyone who contributed and participated for their patience with digital breakdowns, but most of all for their enthusiasm, their curiosity, their highly constructive feedback, and the many inspiring conversations about online manipulation.

Nor from mine own weak merits will I draw The smallest fear or doubt of her revolt, For she had wifi and chose me.

— Revised passage from Othello, Act 3, Scene 3

1 Modern-day Iago

Shakespeare’s Othello depicts a paradigmatic case of manipulation: Iago is jealous of Othello’s relationship with Desdemona and forges a deceitful plan to tear them apart by making Othello believe — falsely — that Desdemona is cheating on him. Amongst other things, he places a handkerchief in the luggage of one of Othello’s close confidants that Othello gave as a gift to Desdemona. Upon finding the handkerchief, Othello falls for Iago’s trap and believes that he was betrayed by Desdemona. Iago’s plan succeeds: a clear case of interpersonal manipulation.!

Interpersonal manipulation can also happen online. A modern-day Iago may have arranged for Othello to find misleading but suggestive messages on Desdemona’s social media account to achieve the same effect. Or he may have harnessed more sophisticated technological means to manipulate messages exchanged between Othello and Desdemona through their voice assistant or smart fridge. And perhaps, there are new forms of interpersonal manipulation that an online modern-day Iago could realise, for example moderating and influencing what people see online and which content they are exposed to. Manipulation is as old as the history of mankind. And yet there are important reasons to be especially concerned about manipulation taking place online, in particular the scale and the nature of online manipulation. First, the scale: what is perhaps most striking about the online world is our increased interaction with algorithms and (autonomous) machines. One editor of this volume, for instance, has screen time warning pop-ups installed but happily clicks Ignore warning for today in order to continue scrolling on Twitter and Instagram. The other editor deleted emails from their phone but simply keeps logging back in through the browser. Worldwide, people spend about two and a half hours on social media every day

(people in the Philippines winning — or losing — the match with a whopping three hours and 53 minutes).? Netflix has 72.9 million users on average, and YouTube almost 200 million, with 80% of US parents of children of ages 11 and below indicating that their kids watch YouTube.* Almost 40% of the US population uses voice assistants.* Most important of all, when it comes to scale, is the breaking down of the online/offline or real-life/digital life boundary, given that our lives are becoming increasingly immersed with (online) technologies.

Of course, the pervasiveness of technology in our daily lives and how technologically blended our lives have become is, as such, no reason to think manipulation must be everywhere, too. It is, however, a reason to be especially alert in light of the tremendous influence that technology seems to have on us. The modern-day Iago is not the CEO of Google or Alibaba per se; Iago may also be hiding in our smartwatch, our Wifi-controlled lights, our robot vacuum cleaner, our care and sex robots, our children’s smart dolls, and our pets’ remotely controlled food machine. So yes, manipulation has always been around, and we’ve known billboards and dubious salesmen for a long time. Right now, however, looking at how our interactions are shaped online, we appear to be dealing with salesmen on steroids and billboards that follow us around and that change depending on who’s looking at them.

A second reason to be especially concerned with modern-day Iagos concerns the nature of online manipulation. Iago is a bad and cunning person, but at least we can understand, conceptually, his cunningness to some extent and have some sense, morally, how to evaluate his actions when his evil ways are brought to the surface. Human manipulation can be just as awful — perhaps even more awful — than technologically mediated manipulation, but we typically know, who manipulated us, and which moral-emotional responses would be (very roughly) appropriate.

All of this is very unclear when it comes to being manipulated by You- Tube videos, voice assistants, personalised Google search results, Candy- Crush, political parties-using-Facebook, and so on. It is often unclear that we are manipulated. Online manipulation is rarely “brought to the surface.” Whereas in Othello there is Emilia who, in the end, uncovers Iago’s manipulation, there are not many online equivalents of Emilia in the digital age. The question of “who” manipulated us is even harder to answer, if that question makes sense at all. And rather than disappointment or anger that many of us experience in light of human manipulation, the typical moralemotional response when one is subject to online manipulation is either confusion, a feeling of powerlessness, or simply indifference or fatigue (“ah, another scandal”). The type of agency and intentionality (not) exhibited by algorithms and more advanced online machines is complex and unclear, making societal-philosophical questions about their manipulative potential all the more acute. This volume aims to address these and other questions about the conceptual and moral nature of online manipulation. Here, we will discuss the aim of this volume in some more detail and provide an overview of the chapters.

2 This volume

Behind the recent public and academic “techlash” seems to be the growing concern that the influence exerted on us by algorithmic systems or more advanced technological machines like robots can be distinctly manipulative and for that reason especially problematic. At the same time, the debate about online manipulation rests on philosophically vexed and, to some extent, underexplored territory. Philosophical attention to manipulation is luckily on the rise (see, for instance, Coons and Weber 2014), and scholars have begun to explore how manipulation differs (or not) from coercion, persuasion, nudging, and other forms of influence, as well as whether manipulation necessarily constitutes a moral wrong of some kind, and, if so, why. The existing literature is still relatively scarce, however, and when it comes to the literature on online manipulation, it is often simply stipulated or suggested, rather than argued for, that a certain technology or technological development is manipulative, and it is sometimes just assumed that because its manipulative it must therefore be morally problematic. However, manipulation might well in some cases be morally unproblematic or indeed desirable, so the inference from “manipulative” to “immoral” is not always evident. Also, various technologies, actions, or developments might turn out to be morally problematic not because they are manipulative but because they are coercive (say). Finally, it is not always clear whether some technological tool or online design would be immoral rather than merely (very) annoying for internet users.

All in all, many fundamental questions about both the nature of online manipulation and its normative status deserve more systematic attention. For instance, must online manipulation (always) involve “intentions” of some sort, and is such a thing as manipulation by a non-human agent possible? Is online manipulation necessarily opaque, or can one be manipulated online “out in the open”? As for questions in the normative domain, is online manipulation always morally wrong, and if so, why? Can online manipulation also be morally acceptable or even a morally good thing to do? Does being manipulated online threaten autonomy, and if so, what do we take autonomy to be?

This edited volume aims to fill a critical gap in current discussions regarding the conceptual nature and moral status of online manipulation. We aim to provide theoretical and normative depth and nuance to debates in digital ethics about the manipulative influence of algorithms and autonomous systems. Thereby, we aim not only to enrich “applied” debates about online manipulation by bringing in contemporary developments from the philosophical debate regarding manipulation but importantly to also enrich and sharpen the philosophical debate by putting existing theories to the test by applying them to online cases and contexts. Finally, we hope to make a methodological contribution by offering a type of applied philosophy that is solidly anchored in philosophical theory whilst strictly in the service of contributing to contemporary societal questions and challenges.

3 Overview of chapters

This volume is the first to explicitly address the philosophy of online manipulation. It contains 20 previously unpublished chapters and brings together leading international philosophers and several promising scholars at earlier stages in their careers. We sought to illuminate the questions surrounding online manipulation specifically from a perspective informed by moral and political philosophy. The chapters in this volume fall under the following four parts: