Danielle Citron speaks at WikiConference USA 2015

Hate Crimes in Cyberspace

It’s terrific to be here. I was saying to someone earlier on, like, I wanted everyone to move up, but no, but this just right. [laughter] I can see everyone.

So it’s terrific to be here, especially given that your community is self-policing. So much, I think we always look to at least–everyone is not a lawyer–looks to lawyers and folks in tech to say, “law, you fix it”, right? and lawyers always say, like, “we want the CS or computer scientists, we want the technologists to fix a lot of our problems”, right. We’re all pointing at each other. But I think, you know, you are all engaged in the process of. sort of, sorting through what your norms are on your site.

So, I’m excited to be here to talk to you about my book, which focuses on the problem of online harassment and stalking.

Online harassment begins: the story of Kathy Sierra

What’s really interesting is I started this project, I would say, about seven years ago—not the book, but just working on the issue—when Kathy Sierra was first targeted online. At the time, when I was–and I’m sure everyone know who Kathy Sierra is, right? Like the technologist? No? [unintelligible voice from audience] Really? Okay, I love that. Right.

So Kathy...she is a technologist who had a blog called Creating Passionate Users. And at the time, like in 2004, 2007, I think, it was in the top, like, 50 top blogs – or most trafficked at the time. She was kind of a trailblazer in her own right. She’s written books on Java and how to code, I mean, clearly uncontroversial. That is, what she’s writing about was how can we code and how can we make people happy, you know, in creating software. And she was targeted on her blog, in the comments and also in email that was sent to her, as well as group blogs.

So, rape and death threats, really graphic, on her own blog, and then an email sent to her. In group blogs, her face appeared in a doctored photograph–and it shows a noose beside her neck. And commenters wrote, “the only thing that Kathy Sierra is good for is her neck size”. And another photograph—she’s very distinctive, beautiful blond woman, and you see her face and its covered by — she looks like she’s being suffocated by lingerie. Now at the time, she was supposed to give a talk–this all happened in a two-week period–kind of this perfect storm of what she understood as anonymous abuse. She had no idea who it was coming from. She cancelled her talk at a tech conference in San Francisco, and she blogged about it. And she said “look, I’m terrified. I’m afraid to go to this conference, I’ve gone to the Colorado police”, where she lived in Boulder. And shockingly, they took her seriously, which is a struggle, even now, right. And they said, don’t leave your house, don’t do anything.

And when the press wrote about her reaction, and what was going on, a sort of second wave of abuse followed, led by people like Weev and others at 4chan, who said “stop whining, Kathy Sierra”. And so they doxxed her, spread her social security number all over the internet, her home, all of her personal contact information, like a narrative that was fabricated about her life and career. And she just got offline. And she really hasn’t blogged since.

And so, Kathy’s struggle and story—she’s sort of one of the main three people I talk about in my book—but she’s emblematic of so many people, who experience online stalking, and at the time when I first started talking...this is why I started talking about Kathy Sierra, we got me off my, sort of, script or whatever.

The law recognizes harassment as a civil rights issue

You know, the idea that we would bring law to bear against threats—privacy invasions like the spreading of social security numbers, that can aid and abet identity theft, defamation—the idea that we would bring law to bear against it was...I was told I was going to break the internet, right? The idea that, like, we could regulate some of this behavior, which criminal and tortious, and as I sort of contend, is a civil rights violation, was really heretical to suggest, and I was totally out of the mainstream. I was told, like, I want to break the First Amendment. And what’s interesting is that seven years later, is this completely–not only is it not heretical to suggest that we should bring law to bear against the worst abuses, but the Electronic Frontier Foundation, ...my friends put up a post in January that says, they said “we agree”.

This is a digital rights problem. That is, the victims have a right to speak and express themselves And this...threats and privacy invasions and defamation...are preventing them from expressing themselves. And that was kind of a moment for me, I have to say, I was, like “Ahhh, you know, my goodness,..” It went from being an off the wall idea, that this abuse is unexceptionable, to being something that civil rights liberties, you know, civil liberties organizations were agreeing with, and said yes, this is a problem of civil rights and civil liberties, right? So mainstream that we were on John Oliver. This is so uncontroversial now, it’s kind of amazing to talk about it.

Scope of speech

So, what I thought I’d do is describe the phenomenon of exactly what I’m talking about — define it, sort of clarify it, like who does it impact, its harm. And then talk a bit about the law, and then what law really can’t do, and a bit about what companies are now doing, right, private entities, and the kinds of choices they’re making. And what does that mean for our system of free expression.

Defining internet harassment

[51:03] Okay, so how do we define cyber-harassment and stalking? And I have to say, I read Wikipedia’s description, and I know it’s not necessarily prescriptive, but your kind of...guidelines...and it was really sophisticated. I thought, this is terrific, right?

So what is harassment and stalking? It is a repeated and persistent course of conduct that’s targeted at a specific individual, that is designed to and intended to, and that causes substantial emotional distress, and often the fear of physical harm. And more importantly—like a definition tells us nothing, right—but it’s how it’s perpetrated, and the components of abuse that are really, I think, important to understand.

Three features of harassment

And it’s often a perfect storm of three key features. One, it really attempts to terrorize people, so by threatening them with physical violence, as Kathy Sierra experienced, by impersonating them online, suggesting that they’re interested in sex, and providing their contact information. It involves reputation—harming lies, like defamation, and then often the manipulation of search engines, to ensure that the lies are prominent in a search of someone’s name. It involves privacy invasion, so that privacy invasion may take the form of hacking into someone’s computer to steal very sensitive information, including nude photos, and then the sharing or posting of them online. We can think of...of course–Jennifer Lawrence is a prominent example, but the iCloud hack is not unusual, and it’s not always celebrities. In fact, normally it’s just an everyday person. But it’s also the posting of people’s nude photos, in violation of their confidence and trust, like we see on revenge porn sites. And the last component is using technology to effectively shove people offline, whether it’s a DDOS attack or otherwise, like engaging in manipulation of technologies with brute force to silence people.

Who does cyber-harassment impact?

So, I think that since I gave you a little, sort of, sense of what this abuse looks like, I’m going to give you….

Anita Sarkeesian and Gamergate

So I’m sure you guys know all about Gamergate. [laughter] I know, right, to say that is kind of crazy to this group, right. Normally, like, when I talk at universities and people are like...I’ll have two really shy gamers in the audience who are like, you know [demonstrates hiding face behind hand] willing to admit that they know what it is. So, Anita Sarkeesian is a friend that is in my book, and I think we largely understand the abuse that she faced, but I think it’s important to know it didn’t start this last summer.

So in 2012, Anita announced that she was starting this Kickstarter campaign to have this video series on sexism in video games. And about a week after Anita announced the launch of this project, she started receiving emails and texts with graphic rape and death threats, bomb threats, right? And that same week, as the cyber-mob kind of descends on her, the game “Beat up Anita Sarkeesian” appears online. So if you google it right this second, you will find this game, right, which is, any key that you touch in your computer–she’s a distinctive woman–you know it’s Anita–her face gets increasingly bloodied, and sort of purplish, right?

Now, whoever was doing this, like a set of people, they went after her livelihood, her Kickstarter campaign, right. So her Kickstarter received hundreds of false abuse complaints, suggesting that her campaign was fraudulent, like, her effort to raise money. Facebook, Twitter, and YouTube received reports that her channels were–this is the irony–and I love this moment–“hate speech, spam, and terrorism”. Right? Seriously, friends? Okay, and because she was very well known, they didn’t shut down her profiles. We know happened at Wikipedia, right, to her Wikipedia page, and the kind of monkey business of...kind of...seed it with pornography, and child porn, and it seems like Wikipedians had to shut down the page. And this abuse went on for years.

So I met Anita before all of this happened, and she called me and said “I know you’ve been writing about this, but I have no idea what to do”. I initially urged her to go to law enforcement, and she said, “what do I say? There are thousands of people coming after me?” Like, I can’t imagine their wrapping their heads around getting a warrant for all of these posters, right?

And we know that last summer it reached a fever pitch after the attacks on Zoe Quinn, and I sort of...one example beyond just the sort of avalanche of rape and death threats...with her address and her parent’s address...it sort of escalated this time. The threats were in her in-box: “I’m going to rape you, bitch, I know exactly where you live,” including her parent’s address and her home address. But also, she was set to give a talk at Utah State University and the day she came—so she’s already there to give the talk—the dean’s office received anonymous phone calls that said, “if Anita Sarkeesian speaks this evening, there will be a school shooting worse than Columbine and New Town combined.” Now Anita was already there, so she was like, “All right, I’ll give this talk so long as there’s some police presence and we’re checking people for guns.” But the response to her was, “Well, we have concealed carry laws, so we can’t help you. We can’t check people.” And isn’t that the most ridiculous thing you’re ever heard, right? So, if President Obama was coming to Utah State, or the CEO of Pepsi, you really, seriously think that’s going to be the response? “We can’t check people?” To make sure that they have no guns? So she didn’t speak, And it was, like, directly impeding what she does for a living, right, is speak and talk–it’s her work as a journalist and media critic.

Low profile people: Holly Jacobs

[57:04] But of course, it’s not just the high profile, whether it’s Kathy Sierra or Anita Sarkeesian. Really, more, I think, importantly, more prevalently, it’s the every day person, right, the grad student, the nurse, the dentist. I can’t tell you the countless people I talked to in research and writing my book. So I’ll just tell you about one, so we can get a full understanding of what this abuse feels like from the perspective of the victim. Holly Jacobs—she had just graduated from BC, and she was in grad school at FIU, and she had a long distance relationship with someone. And they, as young people do (I feel like I’m forty-six; I’m allowed to call her a young person), there were cameras everywhere, and they shared an intimate images with each other. It was a two-way street, it wasn’t, like, one way. They shared images and videos with each other. She was surreptitiously taped during sex too, which she did not consent to, but much of it was consensual sharing. But it was on the understanding, of course, that it was for their eyes only.

So, they break up, and about six months later, she starts getting emails from strangers, and texts, that say “I saw your ad, I’m interested in sex, like, where can we meet?” And at the time—so this is 2009 when it starts. Holly wasn’t in the habit of googling her name, like, it just wasn’t something that we preached and talked to our kids about, and discussed. [58:34] And so, what she found when she did google her name, was over three hundred sites—revenge porn sites, porn sites, and adult finder sites—with her nude images, some videos, and one video of her masturbating, which if you can only imagine the, sort of, sheer embarrassment and horror–but with all of her contact information.

So, she had a part-time job–her work address, her work telephone number, her cell phone number, all of her contact information. And some of the posts—because she was a grad student—said “hot for teacher, Holly and her former last name, she sleeps with students”. Other posts were like fake ads, that said she was interested in threesomes, in anonymous sex, you know, others looked like they were solicitations from her. And her dean’s office—dean of students—received anonymous phone calls that said she was sleeping with her students. And her part time employer—she did some consulting work—she got an email that said “if you don’t send me more nude photos, that I will send these photos that I have of you (nude photos) to your employer, because I know where you work”. And of course, she didn’t write back, as I recommended, because you know, to feed the beast and give more photos is, like, extortion—clearly we have a crime going on here.

And so of course the person lived up to the threat, sent the nude photos to her employer, her part-time employer, right. So she left that job, because it was clear from these postings where she worked, the address. She moved out of her apartment, into her parents’ house, She changed her name. She had a very unique former last name, and her dean of students said to her, “look, if you’re going to teach students, the only way we’re going to let you teach students is you really have to change your name. These young undergrads can’t be googling their teacher.”

Declaring “reputational bankruptcy”

Can you imagine, to declare bankruptcy, as my friend Jonathan Zittrain would say “reputational bankruptcy”–to foist that on someone. It seems like an untenable choice to say to her…. And she hadn’t yet gotten her PhD, so these are people who were in control of her doctorate. So she did. And there’s a wonderful—at the end of my talk I’ll tell you what Holly’s doing today. But she is an anti-harassment advocate. She’s running this wonderful organization called the Cyber Civil Rights Initiative, to raise awareness, change laws. They’ve been incredible, so brave. But when we first met, she could barely tell me her real name, and would call me under different phones. She was afraid she would get hacked. So going public was a huge step for her in 2014 and 2013. But she did it, and she’s trying to change the way we understand this kind of online abuse.

Cyber-harassment: the harm

[1:01:22] But if you can imagine, it’s a hard thing to swallow. And victims…so the damage–we can imagine what the damage is–of all this abuse. Victims change their lives. So Anita did–she couch surfed. Holly moved–she changed her name, right?

The harm – employers google you

[1:01:40] We know that the professional costs are incredibly steep. So people lose their jobs, or it’s impossible to get one. Why? Because online searches are our CV. So, we know that over ninety-five per cent of employers tell us that they are searching people in order to figure out if they are going to interview or hire people. And there are companies that are working on mining data online, to identify candidates. So sometimes you don’t have control over this, right. What a Microsoft study found, that over 80% of employers who are searching people, there’s a negative result. And why does that make sense?

From the employers perspective, it makes eminent sense. Do you want a client saying to you, “Why would you hire that person? Did you see in a google search, like, what this person did?” And it’s not that employers believe that individuals have posted their nude photos online by themselves. It’s not that they believe that Holly’s a prostitute or whatever, right? It’s so much safer, and cheaper and easier to hire someone who doesn’t come with baggage. Just that simple. And victims really struggle emotionally. So many told me, like, “it’s so hard not to wake up in the middle of the night and google yourself, because you think, ‘what next shoe is dropping'”.

The harm – silencing

[1:03:07] And it is silencing. This is a perfect community to talk about all the wonderful things the internet offers–all the knowledge creation, and access to knowledge that it provides. But for victims of online abuse, they feel like the more that they engage online, the more provocation it is for their harassers, so they totally disappear. It’s just safer, and it’s going to provoke less abuse. So they lose the opportunity to do all of the things that we’re all engaging in, politically, socially, economically — all the wonderful things that all these network spaces offer, but they can’t enjoy it.

So sometimes I’m often—this was in the beginning of my work, people sometimes say, “Danielle, can you relax, like, stop making a mountain out of a molehill. It’s one or two bad cases.” And the bottom line is, I wish I could say it was just one or two bad cases, but the Bureau of Justice statistics has made clear in a number of studies, that at least it was estimated in 2006–so you can imaging this was a while ago, it’s clearly gotten worse—that over 850,000 people experience stalking via network technologies every year. And the study defines stalking in the way that I initially...remember my first initial definition?...so it’s not a loosy-goosy definition. It was a clear definition of this sort of study. And we know that...so, Pew came out with a report last year that found this is especially so for women in their 20’s. That at least 20 per cent of women in their twenties will experience cyber-gender harassment in the way that I’ve described it.

Profile of abuse tactics

So, we know that the majority of victims of online abuse are women, but men experience it too. And the playbook for men and women is — it’s so strikingly the same. Every time I get another email or phone call, I’m like — it’s as if I press ‘rewind’ and ‘play’ each time. The abuse is sexually threatening and it’s sexually humiliating. So for both men and women, it’s not just any old threat, but it’s either threats of anal rape, impersonation, suggesting for men that they’re interested in rape, or anal rape – anonymous rape – and for women it’s rape threats. It’s the same thing. It’s sexually targeted threats. And it’s not just any old lie or defamation, it’s sexually humiliating lies. So, individuals are accused of having sexually transmitted diseases, right, they have herpes or AIDS or so alleged. They’re accused of being prostitutes and available for sex. And it’s not just any old privacy invasion, because golly, we know we can invade privacy in so many sundry ways, but it’s often either the theft and hacking and then posting of nude photos that were shared in confidence — the breach of that confidence, and then the sharing online, often alongside personal information.

What can the law do?

[1:06:06] So, it’s a bleak picture now. So, I’m a law professor, naturally I’m going to incline towards what law can do. And it can do something. It can’t solve all of our problems, but it can do something. So what can it do? What can law do right now, and then what are some reforms that I think we should … [phone rings in audience]… Don’t worry, it’s all good, right? (laughs) …What are some reforms that I think we really need and are working on already.

Civil law

[1:06:36] So, what can law do right now? So often victims are told, just sue your harasser, if, of course–if–you can figure out who they are. There are civil claims that victims can bring against their harassers. And it’s absolutely true there are well-tailored torts, like defamation, intentional infliction of emotional distress—[unintellible], privacy tort, public disclosure of private fact. But really, the problem is the practicality of it. It’s so expensive to bring a lawsuit. I always think it’s sort of fabulous and fiery, but not in practical fact. So, it’s so expensive to sue that the civil system, and the kind of corrective justice it could provide for us, is often just unavailable.

Criminal law

[1:07:24] So what about criminal law, right?

At the federal level, which has constrained resources, there are really well-designed federal cyber-stalking — threat, extortion — laws on the books. But the problem is, I think, as the FBI often tells victims, like, “look, this is not an issue of national security, it’s not terrorism, it’s not drugs, we’re not involved”. And they do have their own set of priorities. So, I think to overly rely on the FBI and federal law enforcement is, I think, too tall a task.

[1:08:02] The states have stepped in and long been the first movers, when it comes to sex-stalking and harassment. California was the first in the nation to pass a law against stalking in 1990. And then, within three years, everyone followed, that is, all 50 states. So, you know, we traditionally look to the states. So, about half of the states have really well designed– or they have laws that would reach online abuse. So they have stalking laws, anti-harassment criminal laws, threat laws, laws against solicitation and extortion. In 25 states, we now have laws that criminalize invasions of sexual privacy, when you go after the knowing privacy invader, like the knowing person who posts revenge porn on revenge porn sites, the initial poster.

Problems with enforcement

[1:08:49] But the key problem is enforcement of these laws. So when victims… in describing some of the kind of trivializing that we do to online harassment…. You know, a key problem is that when people go to law enforcement, they’re bedeviled by the same social attitudes that many are bedeviled by, which is “turn your computer off”, is what they’re told. “Oh, boys will be boys.” Like, “Ignore it, it will go away.” And some of that comes from—I have talked to so many law enforcement officers, so it’s not to say that they are, you know, uniquely have really poor attitudes. They’re also just not used to the technology. Like, it’s not a B crime, right? It’s not off-line assault where they say, like, “I’m good at this”. This involves technology, and often they just aren’t familiar with the idea of getting a warrant for an online service provider, to trace a IP address, and then to so produce the warrant to the ISP to figure out if it’s a static IP address–who is this person? They’re like, “whoa, we’ve no idea what we’re doing”. And they also are unfamiliar with the laws, right.

Training law enforcement

[1:10:01] So, we need a whole lot of training. So, it’s work I’m doing with the attorney general of California, Kamala Harris–is training law enforcement. On Wednesday I’m going out to LA to, sort of, announce and roll out this exciting project that we have in training law enforcement. So we have some amazing law enforcers on the beat. But it is really the beginning stages. It’s really the first of its kind in California, and we’re hoping it’s going to catch fire across the country, the training of law enforcement in the laws and the technology. And in getting assistance maybe from the FBI, in training officers, how to approach and investigate these cases.

Inadequate state laws

[1:10:37] But it’s also true that in about half of the states, they just do not have laws on the book, that reach this kind of abuse. So you’re got an aggravated harassment law in New York that only covers communications that are sent directly to victims. So, case from New York: Ian Barber posts his ex-girlfriend’s nude photos on Twitter, he sends them to her employer, and then to everyone in her in-box, including all of her friends, and work contacts. And he’s arrested for aggravated harassment. But rightly so, the court dismisses the indictment against him. Why? Because he never directly sent the harassing abusive communications directly to her. And that makes sense, just the way the legislators wrote that law. So we’re got some work to do at the state level. We’ve got to update harassment and stalking laws that were written in the 90s that are–and some of them are like “email harassment” laws–do we want to have a moment and laugh– or “telephone harassment” laws, which always makes me giggle, right. We’ve got to update those laws; they need to be technology-neutral. And it’s work we can do. We just have to engage some of our state law-makers, among the many issues that they have to worry about. You know, I think we can engage them with this.

Progress with state laws

[1:11:59] You know, we’re seen some progress, 25 states. So Holly Jacobs, the Cyber Civil Rights Initiative, and professor Mary Ann Franks, a colleague of mine at Miami who has been basically working with state legislators across the country. We went from 18 months ago, having one law, New Jersey, that criminalized an invasion of sexual privacy—and that was the law that was invoked to punish Tyler Clementi‘s harasser. Remember the roommate who tapes Tyler having sex with a man... views it...so, surreptitiously watches him having sex with someone and shares it with friends, and he’s then convicted of an invasion of sexual privacy. New Jersey had this law in the books since 2004. Now, 26 states, within 18 months, 26 states have criminalized invasions of sexual privacy. So we’re making some really important movement in the law.

The law: a civil rights approach

[1:12:48] The, kind of, court argument in my book for the way to understand the laws, not only that...look, what we’re seeing is torts, and we’re seeing crimes, but we should fundamentally understand the abuse as a civil rights violation. Why do I say that?

You know, we always think of civil rights violations, or have long thought of them, is when you interfere with people’s substantial and tangible life opportunities. And you do that, because of their membership in a protected group. So think about Anita Sarkeesian. Like, what did the cyber-mob know about Anita? But that she was a woman writing about sexism in video games, truly. And for that, the wrath of the rape threats, privacy invasions, defamation, intimidation, and attempts to basically stop her from doing her work. DDOS attacks aimed at Feminist Frequency, her website, shutting it down intermittently over the years, So, you know, Amanda (Hess) has said really eloquently in Slate—she was writing a piece for Pacific Standard—she writes for Slate. But she had this piece about online harassment in which she said, “Rape threats—what they say to all women, is that they can’t be at ease online.” And I think that that’s exactly right.

Online harassment is often designed to basically make people unemployable, right, and unable to express themselves, which is a fundamental liberty and right. And we have some work to do with our civil rights laws. But I think once we wrap our head around what this is, the law can do only so much. We can only enforce it only a little bit. We can’t [have] perfect-- and frankly don’t want perfect-–enforcement. But what the law does is teach us, right, it’s an educator. And it can help teach us that it’s the wrong thing to do, to help change both social attitudes and behavior.

What that means for free expression

The First Amendment, free speech, and unprotected speech

[1:14:48] So how is any of this, sort of, legal solution consistent with the First Amendment? So, I initially started by explaining–when I first stated talking about online harassment, I was told I was going to break the internet, that is, the internet as a forum for public discourse. And I think a crucially important lesson I think we’ve all now, with eight, nine, years behind us, is that we know that the First Amendment—as a doctrine—it doesn’t operate in absolutes, right? There are certain categories of speech that we know either receive no protection or less rigorous protection, because they give us so little and they cause so much harm.

Unprotected speech

[1:15:29] So much of online harassment is constituted by speech that has either no protection or very little protection. So, what kind of speech am I talking about? True threats, the defamation of private individuals about purely private matters, crime-facilitating speech like extortion and solicitation, and speech that enjoys less rigorous protection, are laws that protect the privacy of communications. So we think of wiretap laws and laws that intervene on behalf of privacy in our communications, we see that they get less — they’re not going to be subject to strict scrutiny. Why? Because they foster private speech. If we don’t think our communications are private, we’re not going to communicate, right. We’re not going to use our cell phones. We’re not going to share nude photos, right? If the assumption is that inherently there is the assumption of risk that they’re public, it will chill speech. And that’s why the Supreme Court has explained that we can uphold, and we have less rigorous protection of speech when what we’re doing is punishing knowing violations of privacy.