Thisislike a realhonortogetThiobeabletobehereinParis, do a workshoponandmeetpeoplewho I havecorrespondedwithonlineormaybemetbeforeonDhe.
Yes.
So I'm thrilledtobehere, so I mostofallprobablyfamiliarwithmesomeone, causethat's probablyhowyoufoundoutaboutthis.
Butincasewejustjustas a matterquickintroduction, what I doactuallyfulltimeisteachat a programinNewYorkcalled T P.
I'lljustpullupthewebsiterealquick.
Um, thisis a twoyeargraduateprogramatTischSchooloftheArtson.
Actually, that's why I'm hereinFranceis N.
Y.
U s a studyabroadprogramandwhyyouParisand I'm visitingthatformostofthisweek.
Andso, inadditiontothis, I havethisYouTubechannelwhere I makecodingtutorialsthatwheremostpeoplefindmeyet.
Oh, theycan't seemyface.
Oh, becauseif I golikethisAllright.
Sosorrytotheliveviewers.
I Thisisthisisjustthewebcam.
I'm gonnabemovingaround a lot, but I'llgiveyou a momentoflookingatyou, butotherwiseit's gonnabeit's gonnabehitormisssobut, umAndsotheotherthingsthat I workon, I think I thinkareimportanttomentionwhicharerelatedtowhat I'm gonnashowtodayis I helpedtoknowwhattherighttermis.
Thisiswhere I leastsofarputallthelinksofthethingsthatarerelevant, and I will I willgooverthisinmoredetailin a second.
ButthisthisisprobablythemostimportantlinkforyouWebeditortemplateBecausethethingthat I wanttomentionisthathere I amintheWebeditoroveronthetopleftupthere.
If I clickthislittlearrowsyou'llseeinhere, itsaysmlfivedotminndotJsSo I havebuilt a versionofthe M l fivelibraryjustforthisworkshopwithsomenewfeaturesthatdon't existinthepublishedfirstoffive.
Sothereyouhaveanyquestionsabout, like, just I'm gonnaswitchovertosomeslidesthataregonnakindof, like, gogounderneaththehoodofthis a littlebitmore, ButbeforewemoveawayfromhereOkay, soletmejustclosethiswindowonand I'm goingtosaviThisSoonethingthat I wanttomentionisyearning.
Shewhoisah, colleagueat N Y u N i T p.
Shetaught a coursethispastsemestercalledMachineLearningfortheWeb.
Andit's basicallyit's a sevenweekcoursethatgoesthroughsomebasicsofmachinelearning, differentkindsofalgorithmsandmodels.
Sohere's a bunchofandsobasicallywhat I'm attemptingtodointhisworkshopiswhat's hereinweektwo.
Thisparticularmodelcanrecognize a varietyofspeciesoflizardsandreptiles, apparently, butitcan't recognize a person.
Sosothat's onethingthat's important.
Whileit's usefulandfuntoplaywith, andit's a nicedemonstrationoftheideaofimageclassification, it's notnecessarilygoingtobereallyusefulforyoutousewithyourownprojectunlesswhatyouhappentodoisyouwant a projectaboutrarebirdspecies, thenmaybethemobilethatmodelbyaccidentbeveryusefultoyou.
ButwhatitactuallydoesOneofthethings.
Oh, yes.
Sothisisthelistoftheclasses.
Sothereissomething.
However, youprobablydon't realisticallyhaveaccessto a databaseof 50 millionimages.
Solet's saywhatyouwanttodoisrecognizethattheexamplethatyouNinghasmadeandthen I'm gonnadosomethingwiththis, likerockpaperscissorsgame, which I learnedtodayisPierreVoices.
Oh, yeah, I wouldhavesaidWalshwere, bythewaytoturn, butapparentlyso.
Yeah, sorockpaper, scissors.
Soyouwouldyoulikeifyouwanttotrain a machinelearningmodel, RecognizeyoumakingtherockgestureorthepapergestureofthescissorsGestureMobilenetsNotgonnadothatbecauseit's notable.
Butif I were a neuralnetworkinonewaytodescribeitisright.
Iftheimageistheinputandtheverylastthingthatcomesoutis a listofprobabilitieswhich I'm showingyourighthere, thereareactually a wholebunchofstepsinbetween.
Sothisisbasicallythe a normalizedpredictionsspectrum.
Itsoundslike a veryfancyword, butbasically, thisisthisisbasicallylookingatalmostlike a raidingoff a numericratingforhowlikelytheparticularimagesAnyoneofthesecategoriesandyoucanseethatyouseethesehighspice.
Solookingathowdataappearsneartootherdatainspaceis a measurementofsimilarity.
Soif I weretobasicallylook, thesetwoareprobablyprettyhere, eachotherin 1000 dimensionalspace, right?
Andthisiswhatwecan.
ThisisSotheprocessoftransferlearningisbasicallyusemobilenettogetin 2000 numbersfromanimageandthatfindsonthenthealgorithmthat I wanttoshowyouisthiscaneyears.
DearestKing, here's neighborKingnearestnearestwhichbasicallyis a wayofsaying, OK, whichcategoryissomething?
Itissomething a memberofbasedonitsproximitytootherentitiesinthatsamespace.
Sowegoallthewaybacktothatteachablemachinesjustfor a secondNow, Um, if I goallthewaybacktohearwhat I'm basicallysayingishere's a wholelotofexamplesofmein 1000 dimensionalspace.
Here's a wholebunchofexamplesofnobodyin 1000 dimensionalspace.
Andnowhere's a newimagelookisn't closertoimagesthatit's alreadyknowsaboutofmeandthepictureorwithmenotinthepicture.
Andthisnewimageisclosertoallofthose.
But I'm not.
If I wanted, like, trytocompareeverysinglepixelcolor, I mightbeabletosortofgetsomethinglikethistowork.
I'm jumpingaroundherequite a bit, certainly, butthistechnique, themobile, thatmodelisjust a digitalfilewith a lotoflikeinformationnumbersinit.
Andthat's a thingthatyoucoulduseinpythoninthebrowserandthentotheconceptoftransfer.
Learningcanbeappliedinanyprogramminglanguagewith a lotofdifferentmachinelearninglibrariesorfromscratchifyouhave a fewyearstowriteeverybitoftheAlbertayourself.
Isthefactthatthisis a waygetimageclassificationsystem, workinginrealtimeinthebrowser, whichpriortothispriortothejobscripttensorflowversionand a lotofthesemodelslikeGlobalNetbeingmadeJohnsoncompatiblewouldbethingsthatyouwouldneedmorepowerfulcomputersarelotsmoretrainingtimetobeabletodo.
Okay, solet's seewhat?
Umsoletmeshowyouactually a reallyniceexampleofhowthisisbeapplied.
SothisisoneofthetensionflowJsexamples.
Umand I'm justgonnaclickonithere, solet's see.
Thisis, uh, yes.
Allowusingcamera.
Okay.
So I'm gonnamakethis a littlebitbigger.
Thisisbasically a projectthatsomeonemadeapologiesthat I don't knowthenameofoneoftheofficialtensorflowjazzexampleswherewhat I coulddonowis I cansayOkay, um, let's say I Pacmanhasfourcontrols.
Sowhat I coulddoisbasicallycreate a physicalcontrollerfrommyself.
ThiocontrolthegameofPacMan.
If I train a train, a setofimagesforboth, right, leftupanddown.
So, uh, I'm gonnajustsaythat, uh, mejustlookinghereregularisIsthisso I'm gonnaclickhere.
It's probably 00.0 a bunchofzerosbeforeitgetstosomeactualnumbers.
Butit's theamountoferror.
Soit's beentraining.
Andhowdoesitknowtheamountoferror?
Well, it's traineditselfwithalltheseimages, knowingwhichonesareright, leftupanddown, andthenitalmostlikeWell, eventhough I knowtheanswers, I couldalsotrytoguesstheanswers.
Sothenlookedatthoseimagesagain, tryingtoguesswhattheanswersareseedsin a guestofthecorrectitifitdidn't get a singleonehome.
It's 100% correctly.
It's airwouldbeeasier.
Um, okay, sonow I'm gonnaclickplay.
We'regonnaseeif I canplaythis.
Where?
What?
I wanttogototheleftfirst.
Maybeithasn't startedyet.
Left.
Okay, up, Up.
Okay.
Whatif I wanttowritedown?
Yougottheidea?
I'm notsurethisis a left.
SoSothepointofthisandanotherprojectthat I willmentionwhichisinthispresentationtowardstheendis, uhthisisposting.
It's different.
I'm gonnatalkaboutisthisisbyandan I t.
P a researchernamedAlejandrocalledPongml.
I won't playthisthreerightnow, butthisis a similarideawherehetrained a teachablemachineforcertaingesturestomove.
Pongpaddlesup, down.
Sothisis a prettythere's hopefullyyoureye.
Yourbrainsarefillingwithideasbeyond, justlike, Oh, I couldtrainthistobelike a weird, gesturalcontrollerfor a simpleto d game, butthisis a nicewayofdemonstratingwhatisthecreativepossibilitiesofwhatyoucoulddowiththiskindoftransferlearningtechniqueandthis?
Sothere's theotherthing.
I thinkit's interestingtomentionhereisthatyou'llnoticethat, like I justtrainedthisrightnow.
Sointheory, a couplethingsoneis I could I couldsaveallofthesetrainingimagesandtheresultingmodelandthenloadedagain.
Likeif I refreshthispage, allthatallthatworkisgone.
However, youokay, soletmeseewhat I Whatdid I misshere?
Solet's see, whatelseisin, uminningspresentationthat I wantedtoYes, I'm I'm gonnatalkaboutposedyet, but I wanttobeconscientiousaboutthetimehere, soletmelookhereandseeifthere's anythingimportantherethathemissed.
Observableobservableis a, uh, systemforcreatingkindoflikeinteractivejavascriptnotebooks.
Onewayofdescribingit's builtby a bunchofpeople, but I thinkstartedbyMike.
Bothstockisthecreatorof a jobsuppliedrecalled d threeandsoMcKeel.
Thisishowyouwouldbasicallycreate a teachtheteachermachinedemowithtensorflowdotJs, and I'm hopefullygonnashowyouaneasierwayofdoingitwithmlfive.
Butthereasonwhy I wanttobringthisuphereisthisis a nicealsodemonstrationofcaptainnearestneighborworkersintwodimensionshere, andyoucanseeit's tryingtoguessismymouthmousepartoftheblue?
I guessthat's kindofblue, purple, bluegrouporredgroup.
AndbasicallythereasonwhyTheobaldGrahamiscalled K nearestneighborsclassifiesitaccordingto a votingsystem.
Soinotherwords, forthisnewpoints, it's lookingforits K nearestneighbor K.
Beingthree, twoofitsnearestneighborsareread.
Onlyoneofitsnearestneighborisblue.
Therefore, itismorelikelytobeoff.
Thecategoryismorelike a cat a dog, butyoucanseethatthisis a kindofalgorithm.
First I couldaddthis, whichisshowingyoulikeactually, whereisthatdecisionboundarythat's everatanypointinspacethat's onthissideisgoingtobeclassifiedaspartofthisredcategory, andthenyoucouldalsoseehowthisworks.
Sothisis a particularalgorithmthat's a classicmachinelearningalgorithm, K nearestneighborthatcanbecomputedvery, veryquickly, whichwhichmakesitpowerful.
Yes, rightif I turntothesidetriestomakeitsbestguest, butitsmostaccuratewith a welllitfullbodyin, um, sowhatit's doingisit's basicallymaking a biglistof X and Y values.
Here's the X Y valueforleft, righthand, therightelbow, therightshoulder.
I couldbasicallysaylike, letmemaketwocategorieshere.
Letmesaythattheseare a bunchofexamplesofmewithmyheadtotheright, Andhere's a bunchofexamplesofbetomyheadwiththeleftandnowyoushouldsee 100% B 100% a bookandgoodbyescreens.
Whatjusthappenedthere?
WhythatjustdecidedThiocompletelydie, but I'lldoitagain.
Left.
I think I tooktheotherwaybeforesoyoucouldseeit's sosoThis.
I wanttoleavesometimeforpeopletojusttrytoplayaroundwithit a bitontheirown.
So I'lltrytogetusfarwith.
Thisis I can, inthenexthalfanhourishandthenwecanalwaysfallbackonthisorcremateexample.
Willyoujustgivethat a try?
ButsoSo I'm gonnaYoucanfollowalong.
I amgoingThio.
Thefirstthingthat I wanttoaddisjustthevideolivevideo, So I'm gonnacreate a variablecalledvideo, andthen I'm gonnasayvideoequalscreatecapturevideo.
I thinkthat's whatisLetmerunthis.
Soifyou'veneverusedthe P fivewebeditorbefore, it's reallyjustThisisthetexteditorwhereyoucanwriteJavascriptandyoucanstopandrestartyourprogramofthesestopandplaybuttons.
Youcanalsoclickthisautorefresh, andit'llupdateit'llrerunthesketchasyourtyping, but I'm gonnakeepthatoffrightnow.
There.
Yes.
Yes.
Nowyouseemenexttome, probably.
Umokay, sonowwehavethevideo.
Now, onething I wanttodois I actuallyjustwantto, um I wanttomakethissmallerand I wanttojust I thinkit's gonnabeThisisnotreallyimportantforthemachinelearningaspect, but I justwanttotakethisvideoanddrawoncampus.
Sothat's thisdrawingcanvas.
Which p fiveThisthroughmydefaultsetuptoworkwith, umandso I amgoingtosayimagevideo 00 I'm gonnachangethevideossizeto 3 20 to 40 Sonowyoucanseethevideoisintwoplaces.
Sothreetimes.
Sonow I'm gonnasayvideodothide.
Andsonow I justhavethevideo.
Sothisisjust a fewsteps, a fewbitsofcodetojustgetthevideodrawinginthecampus.
Okay, Canyouwillseethisfontsize.
Okay.
Okay.
So I'm justgonnatrytopause, but I'm gonnamovefairlyquicklythroughthisstuff.
Uh, so I'm gonnacallthis, uh, featurescalledfeatureextractor.
Sayfeatureextractorequalsmlfivedotfeatureextractor, thenSothefeatureextractoranditisn't really a diagramisthefiveclassthatknowshowtogetthisarrayofthoseknives.
That 1000 dimensionallistof 1000 numbersfromanimage.
Butheknowshowtodothat.
Basedon a preexistingmodel, thatpreexistingmodelismobilenet.
So I'm gonnaIntheory, wecouldapplythesametechniquetononmobileothermodels.
It's a specialkindofobjectcalled a tensor, andbytheway, that's whytenseairflowiscalledtensorflow.
Theideaof a tensoris a fancywordforanarrayofnumbers, butoffanydimension, soitcouldbe a multidimensionalarray, twodimensionalmakedregsofthreedimensionalarray.