is to ensure user trust,

protect user rights and keep users safe,

both from each other

and, at times, from themselves.

Let's talk about what scale looks like at Twitter.

Back in January 2009,

we saw more than two million new tweets each day

on the platform.

January 2014, more than 500 million.

We were seeing two million tweets

in less than six minutes.

That's a 24,900-percent increase.

Now, the vast majority of activity on Twitter

puts no one in harm's way.

There's no risk involved.

My job is to root out and prevent activity that might.

Sounds straightforward, right?

You might even think it'd be easy,

given that I just said the vast majority

of activity on Twitter puts no one in harm's way.

Why spend so much time

searching for potential calamities

in innocuous activities?

Given the scale that Twitter is at,

a one-in-a-million chance happens

500 times a day.

It's the same for other companies

dealing at this sort of scale.

For us, edge cases,

those rare situations that are unlikely to occur,

are more like norms.

Say 99.999 percent of tweets

pose no risk to anyone.

There's no threat involved.

Maybe people are documenting travel landmarks

like Australia's Heart Reef,

or tweeting about a concert they're attending,

or sharing pictures of cute baby animals.

After you take out that 99.999 percent,

that tiny percentage of tweets remaining

works out to roughly

150,000 per month.

The sheer scale of what we're dealing with

makes for a challenge.

You know what else makes my role

particularly challenging?

People do weird things.

(Laughter)

And I have to figure out what they're doing,

why, and whether or not there's risk involved,

often without much in terms of context

or background.

I'm going to show you some examples

that I've run into during my time at Twitter --

these are all real examples —

of situations that at first seemed cut and dried,

but the truth of the matter was something

altogether different.

The details have been changed

to protect the innocent

and sometimes the guilty.

We'll start off easy.

["Yo bitch"]

If you saw a Tweet that only said this,

you might think to yourself,

"That looks like abuse."

After all, why would you want to receive the message,

"Yo, bitch."

Now, I try to stay relatively hip

to the latest trends and memes,

so I knew that "yo, bitch"

was also often a common greeting between friends,

as well as being a popular "Breaking Bad" reference.

I will admit that I did not expect

to encounter a fourth use case.

It turns out it is also used on Twitter

when people are role-playing as dogs.

(Laughter)

And in fact, in that case,

it's not only not abusive,

it's technically just an accurate greeting.

(Laughter)

So okay, determining whether or not

something is abusive without context,

definitely hard.

Let's look at spam.

Here's an example of an account engaged

in classic spammer behavior,

sending the exact same message

to thousands of people.

While this is a mockup I put together using my account,

we see accounts doing this all the time.

Seems pretty straightforward.

We should just automatically suspend accounts

engaging in this kind of behavior.

Turns out there's some exceptions to that rule.

Turns out that that message could also be a notification

you signed up for that the International Space Station is passing overhead

because you wanted to go outside

and see if you could see it.

You're not going to get that chance

if we mistakenly suspend the account

thinking it's spam.

Okay. Let's make the stakes higher.

Back to my account,

again exhibiting classic behavior.

This time it's sending the same message and link.

This is often indicative of something called phishing,

somebody trying to steal another person's account information

by directing them to another website.

That's pretty clearly not a good thing.

We want to, and do, suspend accounts

engaging in that kind of behavior.

So why are the stakes higher for this?

Well, this could also be a bystander at a rally

who managed to record a video

of a police officer beating a non-violent protester

who's trying to let the world know what's happening.

We don't want to gamble

on potentially silencing that crucial speech

by classifying it as spam and suspending it.

That means we evaluate hundreds of parameters

when looking at account behaviors,

and even then, we can still get it wrong

and have to reevaluate.

Now, given the sorts of challenges I'm up against,

it's crucial that I not only predict

but also design protections for the unexpected.

And that's not just an issue for me,

or for Twitter, it's an issue for you.

It's an issue for anybody who's building or creating

something that you think is going to be amazing

and will let people do awesome things.

So what do I do?

I pause and I think,

how could all of this

go horribly wrong?

I visualize catastrophe.

And that's hard. There's a sort of

inherent cognitive dissonance in doing that,

like when you're writing your wedding vows

at the same time as your prenuptial agreement.

(Laughter)

But you still have to do it,

particularly if you're marrying 500 million tweets per day.

What do I mean by "visualize catastrophe?"

I try to think of how something as

benign and innocuous as a picture of a cat

could lead to death,

and what to do to prevent that.

Which happens to be my next example.

This is my cat, Eli.

We wanted to give users the ability

to add photos to their tweets.

A picture is worth a thousand words.

You only get 140 characters.

You add a photo to your tweet,

look at how much more content you've got now.

There's all sorts of great things you can do

by adding a photo to a tweet.

My job isn't to think of those.

It's to think of what could go wrong.

How could this picture

lead to my death?

Well, here's one possibility.

There's more in that picture than just a cat.

There's geodata.

When you take a picture with your smartphone

or digital camera,

there's a lot of additional information

saved along in that image.

In fact, this image also contains

the equivalent of this,

more specifically, this.

Sure, it's not likely that someone's going to try

to track me down and do me harm

based upon image data associated

with a picture I took of my cat,