I'm not actually part of Google.

I'm at Lawrence Berkeley National Lab.

And I'm sort of going to tell you

how we're using TensorFlow to perform

deep learning for the fundamental sciences

and also using high-performance computing.

OK, so the fundamental sciences--

particle physics, cosmology, and lots

of other things I'll explain in a minute--

make heavy use of high-performance computing

at the center I work at, which is

part of the Department of Energy,

initially for simulation and data analysis,

or traditionally.

But progress in deep learning and tools like TensorFlow

have really enabled the use of higher-dimensional data

and through deep learning enable the possibility

of new discoveries, faster computation, and actually

whole new approaches.

And I'm going to talk about that here,

illustrating with a few examples of stuff

we're running at NERSC.

So what is NERSC?

It's the high-performance computing center

for the Department of Energy Office for Science,

which means we support the whole breadth of science

that the DoE does, which actually includes things like

not just cosmology or what you might think of as energy

research, like batteries and so forth, but also

materials and climate and genomics and things like this.

So we have a huge range of users and a vast range

of projects across a whole variety of science.

And we have big machines.

Cori, our latest machine was number five, and, in fact,

the highest by P flops in the US when it was installed

a couple of years ago.

But things dropped down those numbers,

and now it's number 10 in the top 500.

OK, so we see the use of AI now across the whole science

domain.

So you probably can't see this slide very well,

but this is a take on an industry part of machine

learning, sort of splitting it into supervised learning

and unsupervised learning, and classification and regression,

and so forth.

And there's kind of examples here

that we see across the sciences.

But I'll be mostly talking about particle physics and cosmology

because actually that's my background.

So I'm more comfortable with those examples.

So what we're trying to do in these fields

is really uncover the secrets of the universe.

So this is a sort of evolution from the Big

Bang to the present day.

And there's planets and galaxies and stuff.

And they've all been influenced by, for example,

dark matter and dark energy over this evolution.

So obviously our understanding of this

has come a long way in recent years.

But there's still plenty of mysteries and things

that we don't know about, like the very things I just

mentioned, like dark matter.

What is the nature of dark matter?

And what is the relationship between particle physics that

explains extremely well the very small, and even

everything around us, but yet, breaks down

at cosmological scales?

So in order to answer those kind of questions,

we have huge complex instruments,

such as the planned LSST telescope on the left,

which is going to look at the very big in resolutions

that currently I'm presenting.

And the ATLAS Detector at the LHC

on the right, which is at the Large Hadron Collider,

so at the Swiss, French border, it's

a detector the size of a building.

There's little people there.

You can sort of see them.

And it has hundreds of millions of channels of electronics

to record collisions that occur in the middle every 25

nanoseconds, so a huge stream of data.

So both of these experiments have vast streams of data.

Really the ATLAS experiment has processed exabytes

of data over its time.

And this has to be filtered through a process of data

analysis and so forth.

And so if you get like high-resolution images or very

detailed detector outputs, the first stage

is to kind of simplify these, maybe

build catalogs of objects in the sky,

such as stars and galaxies, or in the case of particle physics

to kind of combine these into what particles might have been

produced, and these lines or tracks and deposits that

have occurred in the detector.

So this obviously also involves a large amount of computing.

So computing fits in here.

But computing also fits in because the way

that these analyses are done is to compare with simulated data,

in this case cosmology simulations that

are big HPC simulations done for different types of universes

that might have existed depending

on different cosmology parameters.

And in the particle physics case,

you do extremely detailed simulations

because of the precision you require

in terms of how the detector would

have reacted to, for example, a particle coming in here.

So here you've got all this kind of showering

of what would have happened inside the detector.

And then from each of these, you might

produce summary statistics and compare one to the other

and what are secrets of the universe,

I guess, such as the nature of dark matter,

or new particles at the LHC.

OK, so you might have guessed there's

many areas where deep learning can help with this.

So one is classification to, for example,

find those physics objects that I showed identified

in collisions at the LHC.

Or, indeed, just to directly find

from the raw data what was interesting and what

was not interesting.

Another way you might use it is regression to, for example,

find what kind of energies were deposited inside the detector

or what were the physics parameters that

were responsible for stuff that you saw in the images

at the telescopes.

Another is sort of clustering feature detection

in a more unsupervised way, where

you might want to look for anomalies in the data,

either because these are signs of new physics

or because they're actually problems with the instruments.

And then another last way, perhaps,

is to generate data to replace the full physics simulations

that I just described, which are, as I mentioned,

extremely computationally expensive.

So I'll give a few examples across these domains, OK?

So the first is classification.

So there you're trying to answer the question,

is this new physics?

For example, supersymmetry, which

is also a dark matter candidate.

So I could give you the answer later

whether that's actually new physics or not.

So here the idea is kind of be--

there's several papers now that are exploiting

this kind of idea is to take the detector,

like the ATLAS detector, and sort of unroll--

so it's a cylindrical detector with many cylindrical layers,

and to take that and unroll it into an image where

you have phi along here, which is

the angle around this direction, and eta this way, which

is a physicist's way of describing

the forwardness of the deposit.

And then just simply put it in an image,

and then you get the ability to use

all of the kind of developments in image recognition,

such as convolutional neural networks.

So here's what is now a relatively simple sort

of convolutional network, several convolution and pooling

layers, and then fully connected layers.

And we've exploited this at scales,

either taking 64 by 64 images to represent

this, or at the 224 by 224, sort of closer

to the resolution of the detector.

And then you can also have multiple channels

which correspond to different layers

of this cylindrical detector.

And we saw that it kind of works.

So here you have a rock curve of the two class probability,

so false positive rate.

So the first thing to notice is that you

need a very, very high rejection of the physics

that you already know about because that vastly dominates.

So actually, the data's even been prefiltered before this.

So you need a low false positive rate.

And generally, of course, in these rock curves,

higher this way is better.

So this point represents the physics selections

that were traditionally used in this analysis.

These curves here, like incorporating those higher

level physics variables.

But in shallow neural net-- in shallow machine

learning approaches.

And then the blue line shows that you

can gain quite a lot, and this from using

convolution neural networks, which gives you access.

It's not just the technique of deep learning,

but also being able to use all the data that's available.

And then, similarly, there's another boost

from using three channels, which corresponds

to the other detectors.

OK, so I'd just like to caveat a few of these results.

So even though there's obvious gains to be had here,

these analyses are still not currently used

in the Large Hadron Collider analyses.

And part of the reason for that is

because this is trained on simulation, whereas--

and it might pick up on very small aspects of the simulation

that differ from the real data.

So there's several work that's carrying on

from this to look at how to incorporate

real data into what's done.

But also, I think methodological developments

in how to interrogate these models,

and really discover what they're learning

would be kind of useful for the field.

OK, so taking a regression problem now,

the kind of question you might ask

is, what possible universe would look like this?

So this is an image from the Sloan Digital Sky Survey.

So here the Earth's in the middle,

and increasing redshift takes you out this way.

So this is like the older universe over here.

And you can kind of see.

And the histogram is a structure of galaxies' density.

So there's like more-- you can see

that structure sort of appears as the universe evolves.

And that kind of evolution of structure

tells you something about the cosmology parameters

that were involved.

So can you actually regress those parameters

from looking at these kind of distributions?

So the idea here was to take a method that

was developed by these people in CMU, which is to use, again,

a convolutional neural network.

But here, running on a 3D--

this is actually simulated data, so 3D distribution

of dark matter in the universe.

And these can be large datasets.

So part of the work we did here was to kind of scale this up

and run on extremely large data and across the kind of machines

we have at Cori on 8,000 CPU nodes,

which is kind of some of the largest scale

that TensorFlow has been run on in a data parallel fashion,

and then being able to predict cosmology parameters that

went into this simulation in a matter of minutes because

of the scale of computation used.

So, again, it's a fairly standard network,

but this time in 3D.

And actually, there was quite a bit

of work to get this to work well on CPU, for example.

Because that's-- sorry, I should have mentioned, actually,

that the machine that we have is primarily composed of Intel

Knights Landing CPU.

So that's another area that we help develop.

OK, so again, it kind of works.