ANKUR KOTWAL: Today I want to talk to you about machine

learning for game development.

But before we dive straight into that,

I want to take a step back and show you

some of the magical use cases we've seen in consumer apps.

So for those of you that use Gmail,

you may be aware that the spam filter in Gmail

is actually built on an ML model.

And this model evolves over time,

because users can tag something as spam or not spam.

And the model actually adapts over time based on that.

One of my favorite use cases is actually Google Photos.

So if you wanted to fire up Google Photos right now

and go into the Search field, and type in the word car,

it's going to give you back all the images

in your personal library that have cars in them.

That's not because you went and labeled them.

It's not because someone at Google went and labeled them.

It's because there's an ML model behind it that's actually

able to recognize objects, landmarks, locations,

and even your pets.

So it turns out that games and game development

is actually a rich area for machine learning research.

And in 2014, DeepMind joined us and started

using games as a way to do their machine learning research.

So in 2015 they talked about how they use some classic video

games for ML research.

So what we see on the left hand side there is "Breakout."

And what they did is they used a form of machine learning

called reinforcement learning, where the model

itself only had access to the inputs

that it could provide the game, the visuals, so the screen,

and the score.

And the goal was to try and get as high a score as possible.

So it didn't know how to play the game.

It was just working out its own way.

And within a few hours, it was one

of the best "Breakout" players in the world.

Actually, if you look at the strategy it employs,

it creates a gap on the left hand side,

and lets the ball go through the gap,

and let it bounce off the top wall

and clear the bricks itself.

A year later, DeepMind surprised everybody

by building an ML model that was actually

able to play the game of Go and defeat some world

champions, the world champions at the time.

And more recently in the last few months,

DeepMind and Blizzard have been talking about the work

that they've been doing together with "StarCraft II,"

and building an AI that can play "StarCraft II" competitively.

I recommend you go and check it out.

Actually, again, they pitted the AI Alpha

Star against some pro esports players

and were able to defeat them.

So what we're seeing is games are

a great place for ML research.

And we want to be able to find a way that we

can use games in our own applications,

in our own game development.

But not everybody has a team of machine learning experts

like DeepMind does.

And as you can see from the numbers here, less than 2%

of all developers have any machine learning expertise.

And fewer still are deep learning researchers.

But at Google Cloud, what we want to do

is democratize machine learning.

We want to find a way to make ML available to everybody

so that you can innovate and find

use cases where it's useful.

So today, I'm going to be talking to you about how

you can use machine learning for specific areas of game

development.

We're going to start with player experience,

move into data analytics, and talk about game development.

But the important aspect here is,

we're kind of going to go from easy mode to hard mode.

So think of these as difficulty in your games.

So let's get started with player experience.

We've been doing ML research for years.

And what we have done with that research

is exposed it as a set of APIs that you can readily use today.

Now, these are pre-built models based on our vast data sets.

And we've just exposed them as REST API endpoints

where you can consume them either on your server,

or directly through your clients as well.

But these are generic APIs.

And I'll cover a few of them.

Let's look at some specific examples

where game developers could benefit from them.

So we're living in a world where, increasingly, we

have a global gaming audience.

We have players that are able to connect with each other

from vastly different parts of the world.

And language is, frankly, a challenge.

Some game developers have used techniques like emotes

to try and get around this, where

we limit the type of vocabulary that can be used,

and that makes it easy to translate.

But when you're in the thick of battle,

in a battle royale game, and one of your squad members

speaks a different language to the other folks,

it's really hard to coordinate.

So we can do better here.

Now, you may have heard of Google Translate.

It's a consumer application where

we can translate languages from a source language

to a destination language.

You may have seen it in Google Chrome, where

if you go to a website that's in a language that's not

your default language, Google Chrome offers

to translate that for you.

But we've exposed that as an API.

And we call it the Google Translate API.

I'm going to switch to a demo just to show you how it works.

So can we switch to the demo machine, please?

There we are.

So here is actually just the landing page for the API.

cloud.google.com/translate.

And when we scroll down, you'll see that we've actually

got a demo that you can see.

So first thing I'm going to do is click the recapture.

And switch languages.

So the type of thing that we might have players say is, oh,

good game.

Have fun.

And we can do it across different languages.

So let's say we choose Dutch.

You'll see it's responsiveness is incredibly fast.

We can say something.

Just good luck.

Oops.

And, again, we can switch to any language.

What you'll see, though, when I expand

this is that request URL is all that's needed.

We have a couple of parameters, query parameters, where we say,

here's what the source text is, what the source language

is, the destination language.

And then we provide our API key for billing purposes.

That's it.

And what we get back is a nice little bit of JSON that

gives us the translated text.

So if you have any sort of chat messaging in your games,

this is a way that you could translate between languages.

All right.

Let's switch back to the slides, please.

So even though we're enabling people to talk to each other,

people aren't always friendly to each other.

You may have noticed that on the internet.

And what this does is it actually

creates a bad experience for your other players.

When you've got one person dominating the conversation,

or a group of people that are being hurtful to others,

it really causes problems.

And the way developers treat this sort of scenario

these days is that they provide a mechanism for players

to report other players, report bad behavior.

At which stage, you gather a bunch

of diagnostic information, maybe some chat logs,

maybe they're in-game recordings,

and so on, and you pass it off to a team that

has to triage it.

That's a manual effort.

Triaging that sort of work takes a lot of time.

We have an API called the Cloud Natural Language

API, which can actually detect sentiment

in individuals' chat messages.

So as a way of quickly triaging through reports,

you can quickly identify where you

may have some problem areas in your logs,

and make that triage process a lot simpler.

Increasingly what we're seeing in games

these days is that developers are

starting to adopt things like AR,

where they're using the camera.

And they need to be able to detect

what type of objects that are in the scene,

or what type of locations they're in, maybe landmarks.

Another example that we see is lots of game developers

are providing ways for your players

to create user generated content.

It might be items.

It might be custom images.

It might be even maps.

Turns out that if you give people the ability to upload

whatever they like, they can, again,

upload things that are probably not appropriate for everyone

there.

Now, we have an API called the Vision API.

And it is able to do things like object detection,

and also able to flag explicit content.

Now, Vision API is really cool.

Because it's giving you the kind of power that we

have in Google Photos, that example I used earlier,

but giving it to you as an API that you can readily call.

Now, before I switch to the demo,

I just want to get some answers.

Can anybody tell me what that is?

I'm hearing Eiffel Tower.

Any other thoughts?

All right.

So we'll switch to the demo machine.

We're going to look at that image right now.

This is our Vision API landing page, cloud.google.com/vision.

And what we're going to do is drop an image on there,

hit the recapture again.

This is, again, just a demo.

But what we're doing is uploading this image.

And if you said Eiffel Tower, you were wrong.

This is actually the Paris Hotel and Casino in Las Vegas.

Now, when that API returned us a response,

it actually told us what part of the image

that it used to recognize what this landmark was.

And so you can see there's a green highlight,

or a green bounding box around that image.

And the way that it knew that it wasn't the Eiffel Tower,

The Eiffel Tower doesn't have a building below it,

unlike the Paris Hotel and Casino.

Now, because this is a real place in the world,

we're able to get some useful information.

We're able to see that this is a real landmark, where it is.

We're able to get links to the web that says what kind

of information this is.

And we get object detection.

We get labels.

We can see that this is a landmark,

that this is a tourist attraction,

and actually, that we even have a lot of the sky in this image.

And then we get Safe Search.

So we can see what kind of image it was.

I was going to give you an example of an explicit image,

but legal said no.

So we'll have to move on.

All right.

Let's move back to the slides, please.

So that's the Vision API.

And when we look at some of these use cases,

you can see that just by using our APIs,

you're able to solve kind of low hanging fruit, quality

of life type problems.