Anysoftwaredevelopershere, giventhatit's I Oh, sure, onthis, thisisthistransformation.
Thisrevolutionis, particularlyfrom a developersperspective, isreally, reallycoolbecauseit's givingus a wholenewsetoftoolsthatwecanusetobuildscenariosandtobuildsolutionsforproblemsthatmayhavebeentoocomplextoevenconsiderpriortothis.
Soshouldwetake a lookatsomeofthesesofirstofall, astronomyatschool?
I studiedphysics.
I wasn't thecomsidepartsince I'm a physicsandastronomygeekonitwasn't thatlongagowhenwelearnedhowtodiscoverwhat, hownewplanetsaroundotherstarsinourgalaxy, thewaythatwediscovereditwasthatsometimeswouldobservelike a littlewobbleinthestar.
Andthatmeantthattherewas a verylargeplanetlikeJupiter, sizeorevenbigger, orbitingthatstarverycloselyandcausing a wobblebecauseofthegravitationalattraction.
But, ofcourse, thekindofplanetswewanttofindoutthesmall, rockyoneslikeArthurMars, whereyouknowthere's a chanceoffindinglifeontheseplanetsonDDE.
They'veactuallyrecentlydiscoveredthisplanetcalledKepler 90 i bysiftingthroughdataandbuildingmodelsforusingmachinelearningandusingtensorflowonKepler 90.
EyeisactuallymuchclosertoitshoststarthanEarthissothatitsorbitisonly 14 daysinsteadofour 365 and 1/4 in a bitchon.
Notonlythatwhich I findreallycoolthattheydidn't justfind.
I findit's just a wonderfultimetobealivebecausetechnologyisenablingustodiscoverthesegreatnewthings.
Andevenclosertohome, we'vealsodiscoveredthatlookingatscansofthehumaneye, asyouwouldhaveseeninthekeynote, youknow, withmachinelearningtrainedmodelsonthis, we'vebeenabletodiscoverthingssuchasbloodpressurepredictionsorbeingabletoassess a person's riskof a heartattackor a stroke.
Now, justimagineifthisscreeningcanbedoneon a smallmobilephone, howprofoundistheeffectsgoingtobe?
A noninvasivescreeningforthingssuchasheartdiseaseisit'llbesavingmanylivesbutalsobeimprovingthequalityofmany, manymorelives.
Now, thesearejust a fewofthebreakthroughsandadvancesthathavebeenmadebecauseoftensorflowandtensorflow.
We'vebeenworkinghardwiththecommunitywithallofyoutomakethis a machinelearningplatformforeverybody.
Sotodayandwhenwewanttoshare a fewofthenewadvancesthathavebeenworkingonthis.
SoincludingwillbelookingatrobotsonVincent's gonnacomeoutin a fewmomentstoshowusrobotsthatlearnandsomeoftheworkthatthey'vebeendoingtoimprovehowrobotslearn.
AndthenDebbieisgoingtobefromnurse.
She's gonnabeshowinguscosmologyadvancementsonincludingshowinghowbuilding a simulationoftheentireuniversewillhelpusunderstandthenatureoftheunknownsinouruniverse, likedarkmatteranddarkenergy.
Butfirstofall, I wouldlovetowelcomefromthemagentateamwehavedug.
Who's theprincipalscientists, Doug.
Thanks, Florence.
Thanks, Doug.
Thankyouverymuch.
Allright.
Daythree.
We'regettingthere, Everybody.
I'm Doug.
I am a researchscientistatGoogle, workingon a projectcalledMagenta.
I wanttotalktoyou a littlebitaboutmusicandartandhowtouse a machinelearningpotentiallyforexpressivepurposes.
So I wanttotalkfirstabout a drawingprojectcalledSketchAre n n wherewetrained a neuralnetworktodosomethingasimportantasdrawthepigthatyouseeontherightthere.
And I wanttousethisasanexample, actuallyhighlight a few, I thinkimportantmachinelearningconceptsthatwe'refindingtobecrucialforusingmachinelearninginthecontextofartandmusic.
Solet's divein.
It's gonnaget a littletechnical, buthopefullytobefunforyou, allwe'regonnadoistrytolearntodraw, notbygeneratingpixels, butactuallybygeneratingpenstrokes.
And I thinkthisis a veryinterestingrepresentationtousebecauseit's veryclosetowhatwedowhenwedrawsospecifically, we'regonnatakethedatafromtheverypopularquickdrawgameplayingPictionaryagainstmachinelearningalgorithm.
AtthatwascapturedhisDelta X delta y movementsofthepen.
Whatyou'reseeingontheleft, theencoderNetworksjobistotakethosestrokesofthatcatandencodetheminsomewaysothattheycouldbestoredas a latentvector, theyellowboxinthemiddle.
Thejobofthedecoderistodecodethatlateinvectorbackinto a generatedsketchandtheveryimportantpoint.
Butifyouhave a lookatthelightandspaceanalogies, wetakethelateinvectorfor a catheadandweadd a pigbodyandwesubtractthepighead.
Andofcourse, itstandstoreasonthatyoushouldget a catbodyandwecoulddothesamethinginreverse.
Andthisisrealdata.
Thisactuallyworks, andthereason I mentionitisitshowsthatthelateinspacemodelsarelearningsomeofthegeometricrelationsbetweentheformsthatpeopledraw.
I'm goingtoswitchgearsnowandmovefrom, uh, drawingtomusicandtalk a littlebitabout a modelcalledIncense, whichis a neuralnetworksynthesizerthattakesaudioandlearnstogeneralizeinthespaceofmusicyoumayhaveseenfromthebeginningof a Iotwithbathingthathasbeenputinto a hardwareunitcalledInstantSuperhowmanypeoplehaveheardofhandsincesuper?
Howmanypeoplewantaninstant?
Supergood.
Okay, well, that's possible, asyouknow.
Um, okay, so I wantforthoseofyouthatdidn't seetheopening, I have a shortversionofthemakingoftheinstant.
Superliketorollthatnowtogiveyouguys a betterideaofwhatthismodel's upto.
Sotheideathatyou'removingaroundthelateinspace, andyou'reabletodiscoversoundsthathopefullyhavesomesimilarityandbecausethey'remadeupoflearningwhatmakeshumanshowsoundworksforusinthesamewayas a pigtruckgivesusmaybesomenewideasabouthowsoundworks.
And, asyouprobablyknow, youcanmaketheseyourself, which I thinkismymind.
So I wanttokeepgoingwithmusic, but I wanttomoveawayfromaudioand I wanttomovenow.
ThioMusicalscores, musicalnotessomethingthatyouknow, thinkoflastnightwithjusticedriving a sequencerandtalkaboutbasicallythesameidea, whichis, canwelearn a lateinspacewherewecanmovearoundwhat's possibleininin a musicalnoteor a musicalscore?
Rathersowhatyouseehereissomethreepartmusicalthingonthetopandsomeonepartmusicalthingonthebottomandthenfindingin a lateinspacesomethingthat's inbetween, okay?
Andnow I putthefacesunderneaththis.
Whatyou'relookingatnowis a representationof a musicaldrumscorewheretimeispassinglefttoright.
Andwhatwe'regoingtoseeiswe'regonnastart.
I'm gonnaplaythisforyou.
It's a littlebitlong, so I wanttosetthisup.
We'regonnastartwith a drumbeatonemeasureofdrums, andwe'regonnaendwithonemeasureofdrumsandyou'regonnahearthose.
Firstyou'regonnahear A and B, andthenyou'regoingtohearthislateinspacemodel.
Trytofigureouthowtogetfrom a to B, andeverythinginbetweenismadeupbythemodelinexactlythesamewaythatthefacesinthemiddlearemadeupbythemodel.
Soasyou'relistening, basically, listenforwhetheritmakesmusicalsenseornot, thattheintermediatedrumslet's giveit a role.
Soyouhaveitmovingrightalong.
Itturnsout, take a lookatthiscommand.
Um, thismakesensetosomeofyoumaybe, weweresurprisedtolearn, after a yearofdoingmagenta, thatthisisnottherightwaytoworkwithmusiciansandartists.
I know I laughedtoo, butwereallythoughthewas a greatidea.
OK, so, um, we'vewe'vemovedquite a bittowardstryingtobuildtoolsthatmusicianscanuse.
Thisis a drummachine, actually, thatyoucanplaywithonlinebuiltaroundtensorflowdotJsand I have a shortclipofthisbeingused.
Whatyou'regoingtoseeisalltheredisfromyou.
As a musician, youcanplayaroundwithitandthentheblueisgeneratedbythemodels.
Solet's givethis a roll.
Thisisquite a bitshorter.
Sothisisavailableforyouas a codepenwhichallowsyoutoplay a roundoftheHTMLandtheCSSandtheJavaScriptandreallyamazing a hugeshoutouttoTeroParviainenwhodidthis.
HegrabbedoneofourtrainmagentamodelsandheusedtensorflowdotJsandhehacked a bunchofcodetomakeitwork.
Somethingwouldgivethem a rewardandovertimetogetbetterandbetteratit.
Ofcourse, weusedtheplanningforthisum, basicallyhave a convolutionallnetworkthatmapsthoseimagesthattherobot C oftheworkspaceinfrontofthemtoactionsandpossibleactions.
Soontherighthere, youseewhat a typicalsimulationof a robotwouldlooklike.
Thisis a virtualrobotstryingtograspobjectsandsimulation.
Whatyouseeontheothersideheremaylooklike a realrobotsdoingthesametask, butinfact, itiscompletelysimulatedaswell.
We'velearned a machinelearningmodelthatmapsthosesimulatedimagestorealimagestoreallookingimages.
They'reessentiallyindistinguishablefromwhat a rielrobotwouldseeintherealworld.
Andbyusingthiskindofdatain a simulatedenvironmentandtrainingassimilatingmodeltoaccomplishtasksusingthoseimages, wecanactuallytransferthatinformationandmakeitworkintherealworldaswell.
So, attheendoftheday, whatweendupwithisyetanotherbigconvolution, allnetworkthatmapswhatyouseeatthetoptowhatyouseeatthebottomwithoutinvolvinganythree D cameraoranythinglikethat.
Thewaywegoaboutthisisthatifyouthinkaboutimitatingsomebodyelse, forexample, somebodypouring a glassofwaterkindofcokeitallreliesonyoubeingabletolookatthemfrom 1/3 partyofyouandpicturingyourselfdoingthesamethingfromyourpointofview, whatyouwouldlooklikeifyoudidthesameseeingyourself.
Sowecollectedsomeofthisdatathatlookslikethatwhereyouhavesomebodylookingatsomebodyelsedo a taskandyouendupwiththosetwovideosoff, onetakenbythepersondoingthetaskandanotheronetakenbyanotherperson.
We'reseeingthesequestionsshowingupinourworkloadonoursupercomputers, So I wanttofocusononeparticulartopic.
Areaisveryclosetomyheart, whichiscosmology, because I'm a cosmologistbytraining.
Mybackgroundisin a cosmologyresearchbecause I'vealwaysbeeninterestedinthereallythemostfundamentalquestionsthatwehaveinscienceaboutthenatureoftheuniversefrom.
Wehave a fairlygoodfeelforhowmuchdarkenergythereisintheuniverse, howmuchdarkmatterhowmuchregularmatterthereisintheuniverse.
Andthere's onlyabout 5% ofregularmatter, whichiseverythingthatyouand I andallthestarsandallthedustnorthegasandalltheGalaxiesoutthere, they'remadeofregularmatterandthatmakesup a prettytinyproportionofthecontentsoftheuniverse.
Thethingthat I findreallyinterestingiswedosejustdon't knowwhattherestofitis.
Thefactthatthereissomuchthatwehaveyettodiscovermeansthatthey'retremendouspossibilitiesfornewwaysforustounderstandouruniverseandwearebuilding a biggerandbettertelescopeswerecollectingdataallthetime, takingimagesandobservationsofthesky.
Theparasspectrumis a measureofhowmatterisdistributedthroughouttheuniverse, whetherit's kindofdistributedfairlyevenlythroughoutspaceorwhetherit's clusteredonsmallscales.
Thisisillustratedinthis.
Theimagesonthetopofthislighthere, whichissnapshotsof a simulateduniverse, runin a stupidcomputer, andyoucanseetheovertimegravityispullingmattertogetheronsothatstartmatterandregularmatter.
Soyoucanimaginethethreedimensionalvolumecollapsedinto a twodimensionalprojectionoffthemassdensityintheuniverse.
Asyou'relookingoutattheskywhenweusedagainwhichisbasicallyfairlystandard D c gantopologytoproducenewmapsoffthesenewmassmapsbasedonsimulations S O, thisisanaugmentationwe'reusingthisnetwork.
Butas a scientist, kindofsquintingatsomethingandsaying, Ohyeah, thatlooksaboutrightisnotgoodenough.
What I wantistobeabletoquantifythis, tobeabletoquantifyhowarethenetworkisworkingandquantifyhow, liketherialimagesourgeneratedimagesareThisis, I think, wherescientificdatahas a realadvantagecomparedtoyournaturalimagedatabecause, uh, scientificdataofusuallyveryoftenhasassociativestatisticswithit.
Somakingthisinto a trueemulatoronandthiswillhelpreducetheneedfortheseverycomputationtheexpensivesimulations.
And I arecosmologiststoexploreparameterspace a bitmorefreely, and I'd liketoexplore a littlebitfurtherwhatthisnetworkisactuallylearning.
I saw a reallyinterestingtalkthismorningherewastouchingonthiskindofthinghowwecanusemachinelearningtogaininsightsintothenatureofthedatathatwe'reworkingwith.
Esointheworkthat I'm showingherewewerelookingatwhichstructuresinourmassmapsarecontributingtothemodelofmoststronglycontributingtothemodelbylookingat a quantitycalledsaliencyandsobyifyoulookatthemapofsaliency, whichistheblackandwhiteimagehere, youcanseethatthepeaksinthesalientseeMattcorrespondtopeaksinthemassmapondsothesepeaksinthemassAbouttheseareconcentrationsofmatter.
Oneofmythreedimensionsareinterestingfromus, from a computationalpointofview, becausein a threedimensionaldatavolume, you'relookingatthreedimensionalmatrices, threedimensionalconvolutions.
Thisissomethingthat's computational, expensive, andit's somethingthatcanrunreallywellon a supercomputingarchitectures.
So a teamatCMUrecentlydemonstratedforthefirsttimethatdeeplearningcouldbeusedtodeterminethephysicalmodeloftheuniversefromthreedimensionalsimulationsofthefullmatterdistribution.
Sothisisthefullthreedimensionalmatterratherthan a twodimensionalprojectionofthematter.
Densityonthisworkshowedthatthenetworkwasabletomakesignificantlybetterestimatesoftheparametersthatdescribethephysicsoffthesimulateduniversecomparedtotraditionalmethodswhereyoumightbelookingatoneofthesestatistics, likethepowerspectrumondhe, Sothistleis a reallyniceexampleofhowthenetworkwasabletolearnwhat's structuresinthisthreedimensionalmatterfallingmoreimportantratherthanjustlookingatstatisticsthatweinadvancethoughtwasgoingtobeuseful.
Wehave a specializedhardwaretoallowthetensofthousandsofcomputerswehaveonthesesupercomputerstoactstogetherasonecomputermachineyouwanttousethismachineasefficientlyaspossibletotrainournetwork.
Sotheapproachwetakeasusing a fullysynchronousdataparallelapproachwhereeachnodeistrainingon a subsetofthedataonwestartedoffasmanypeopledo.
Youusing a g r p c forthiswhereeachcomputenoticecommunicatingwiththeparameterservertosendtheirparameterupdatesandhavethatsentbackandforward.
Butlikemanyotherpeoplehavenoted, this'llisnot a veryefficientwaytorun a scale.
Wefoundthatifwe'rerunningbeyond 100 notesalso, thenwehad a realcommunicationbottleneckbetweenthecomputenodesontheparameterservice.
SoinsteadweuseMP I, whichis a messagepassinginterfacetoallowourcomputerknowstocommunicatewitheachotherdirectly, soremovingtheneedforparameterserversondhe.
This'd alsohastheadvantagethatcanreallytakeadvantageofourhighspeedinterconnect, thespecializedhardwarethatconnectsourcomputernotesesoweuse a figradiantaggregationforthis.
WeusespecializedMP I collectiveAllreduce, whichisdesignedbyCrais, whoareourpartnerswithoursupercomputersonthisempty I alreadyuseisisprettyneat.