Sodon't beworriedaboutthatnow, before I gettoofarintothisfirstvideo, I justwanttoquicklygiveyouanoverviewofexactlywhat I'm gonnabedoingthroughoutthisseriousYouguyshaveanideaofwhattoexpectandwhatyou'regoingtolearnNow, thebeginningvideos, andespeciallythisonearegonnabededicatedtounderstandinghow a neuralnetworkworks.
And I thinkthisisabsolutelyfundamentalandthatyouhavetohavesomekindofbasisonthemathbehind a neuralnetworkbeforeyou'rereallyabletoactuallyproperlyimplementone.
Now, Tensorfloatis a reallynicejobofmakingitsupereasytoimplementneuralnetworksandusethem.
Buttoactuallyhave a successfulandcomplexnonetwork, youhavetounderstandhowtheyworkonthelowerlevel, sothat's goingtobedoingforthefirstfewvideos.
Afterthat, whatwe'lldoiswe'llstartdesigningourownneuralnetworksthatcansolvetheverybasic M nusdatasetsthattensorvoteprovidestous.
Nowtheseairprettystraightforwardandprettysimple, buttheygiveus a reallygoodbuildingblockonunderstandinghowthearchitectureof a neuralnetworkworks.
Whataresomeofthedifferentactivationfunctions, howyoucanconnectlayersandallofthatwhichwilltransitionusnicelyintocreatingourownneuralnetworks, usingourowndataforsomethinglikeplaying a game.
Now, personally, I'm reallyinterestedwithneuralnetworksplayinggames, and I'm sure a lotofyouareaswell.
Andthat's what I'm gonnabeaimingtodoneartheendoftheSiris, onkindofourlargerprojectwillbedesigning a neuralnetworkandtweakingitsotheycanplay a verybasicgamethat I havepersonallydesignedinpythonwithPieGame.
Inmostcases, whatwedoiswehavewhat's called a fullyconnectedneuralnetwork, whichmeansthateachneuroninonelayerisconnectedtoeachneuroninthenextlayer.
Exactlyonetime.
Soif I weretoaddanotherneuronhere, thenwhatwouldhappeniseachoftheseneuronswouldalsoconnecttothisneurononetimesoitwouldhave a totalofeightconnectionsbecausefourtimestwoisright, andthat's howthatwouldwork.
NowforsimplicitySteak.
We'rejustgonnauseoneneuroninthenextlayerjusttomakethings a littlebiteasiertounderstand.
Nowalloftheseconnectionshavewhatisknownas a wait.
Now, thisisin a neuralnetworkspecificallyOK, sowe'regoingtosaythisisknownasWaitone.
And I talkedabouthowthisworksmoreindepthin a second, butjustjustfallmenow.
SowhatthissymbolmeansistakesomeAh, andwhatwedois I'm goingtosayinthiscase, I whichisgonnabeourvariable, and I'lltalkabouthowthiskindofthingworksin a secondwillsay I equalsone, and I'm goingtosaywe'lltaketheweightedsumof, inthiscasevalue I multipliedbyWait, I Sowhatthismeansessentiallyisgonnastartat I equalsonewe'regonnauseeyesarevariableforlooping.
Andwe'regoingtosayinthiscase, we'regoingtobeonetimes V I r servi I times w I.
Andthenwe'regonnaaddallthose.
Sowhat?
Thiswillreturntous.
Actually, itwillbetheone w onplus b two w twoplus b three w threeplusofthefour w fourandthiswillbeouroutput.
That's that's whatouroutputlayerisgoingtohaveas a value.
Andeventuallyyourgoalisthatyougetto a pointwhereyournetworkisveryaccuratebecauseyou'vegivenit a tonofdataanditsadjustedtheweightsinthebiasescorrectlysothatthiskindofformulahereofthisweightedaveragewilljustalwaysgiveyouthecorrectanswerorhas a veryhighaccuracyorhighchanceofgivingyouthecorrectanswer.
Andthatishowthelearningprocessworksfor a neuralnetwork.
Allright, sonowit's timetodiscuss a littlebitaboutactivationfunctions.
Sorightnow, what I'veactuallyjustdescribedtoyouis a veryadvancedtechniqueoflinearregression.
Soessentially I wassaying, We'readjustingwaits, We'readjustingbiasesandessentiallywe'recreating a functionthat, giventheinputsoflike, X y z w likeleftfront, right, We'regivingsomekindofoutput.
Butallwe'vebeendoingtodothatessentially, isjustadjusting a linearfunctionbecauseourdegreesonLeewonright?
Wehaveweightsofdegreeonemultiplyingbyvaluesofdegreeone, andwe'readdingsomekindofbiasandthatkindofremindsyouoftheformMXplus B.
We'reliterallyjustadding a bunchofMXplusbeestogether, whichgivesuslike a fairlycomplexlinearfunction.
Butthisisreallynot a greatwaytodothingsbecauseitlimitsthedegreeofcomplexitythatournetworkcanactuallyhavetobelinear.
Whenactivationfunctiondoesis, it's essentially a nonlinearfunctionthatwillallowyoutoadd a degreeofcomplexitytoyournetworksothatyoucanhavemoreof a functionthat's likethis, asopposedto a functionthatis a straightline.
Soanexampleofanactivationfunctionissomethinglike a sigmoidfunction.
Ifyouhave, like, a degreeninefunctionthanwhatyoucoulddoisyoucanhavesomecrazykindofcurveah, andstuffgoingon, especiallyinmultipledimensionsthatwilljustmakethingslikemuchmorecomplex.
So, forexample, ifyouhave, like a degreeninefunction, youcanhavecurvesthatairgoinglikelikethis, likeallaroundherethataremappingyourdifferentvalues.
Andifyouonlyhave a linearfunction, whileyoucouldonlyhave a straightline, whichlimitsyourdegreeofcomplexityby a significantamountNowwhattheseactivationfunctionsalsodoistheyshrinkdown, you'redown a sothatitisnotaslarge.
So, forexample, rightlikesay, well, workingwithdatathatislikehundredsofthousandsoflikecharacterslongerdigits, we'd wanttoshrinkthatinto, like, normalizedthatdatasothatit's easiertoactuallyworkwith.
Soletmegiveyou a morepracticalexampleofhowtousetheactivationfunction.
I talkedtoher.
Whatsigmoiddoes?
Whatwewoulddoiswewouldtakethiswaitedsums.
Wedidthepsalmof W I B I, umplus B I writeandwewouldapplyanactivationfunctiontothis.
So I wouldsaymaybeouractivationfunctionisFX, andwewouldsay f ofthisandthisgivesussomevalue, whichisnowgonnabeouroutputneuron.
Andthereasonwedothatagainissothatwhenweareadjustingourweightsandbiasesandwehavetheactivationfunctionandnowwecanhave a waymorecomplexfunction, asopposedtojusthavingthekindoflinear, regressionstraightline, whichiswhatwe'vetalkedaboutinmyothermachinelearningcourses.
Soifthisiskindofgoing a littlebitoveryourhead, itmaybemylackofexplainingit.
Whichmeansthatwe'regonnaadd a higherdegreeofcomplexitytoourmodel, whichwillallowustocreatewaymorecomplexmodelsandneuralnetworksthatconsultbetterproblems.
I don't reallywanttotalkaboutlossfunctionstoomuchbecause I'm definitelynoexpertonhowtheywork.
Weadjustitbasedonwhateverthatlostfunctiongavetouswhenitmeansthatthishaskindofbeenmyexplanationof a neuralnetwork.
I want a very I wanttostaterighthereforeveryonethat I amnoproonneuralnetworks.
Thisismyunderstanding.
Theremightbesomestuffthat's a littlebitflawedorsomeareasthat I skippedover, andquickly, actually, because I youknow, ifsomepeopleprobablygonnasaythiswhenyou'recreatingyourownnetworksaswell, youhaveanotherthingthatiscalledHedidn't layers.
Sorightnowwe'veonlybeenusingtwolayers.
Butinmostneuralnetworks, whatyouhaveis a tonofdifferentinputneuronsthatconnecttowhat's knownas a hiddenlayerormultiplehiddenlayersofneuron.
Soyoucanhavecrazyneuralnetworksthatlookssomethinglikethis, butwithwaymoreneuronsandwaymorelayersandallthiskindofstuff, I justwanttoshow a verybasicnetworktodaybecause I didn't wanttogoinandtalkaboutlike a tonofstuff, especiallycause I know a lotofpeoplethatwatchmyvideosarenotparole.
Mouthguysarejusttryingtoget a basicunderstandingandbeabletoimplementsomeofthisstuff.
Now, intoday's video, whatwe'llbedoingisactuallygettingourhandsdirtyandworkingwith a bitofcodeandloadinginourfirstdataset.
Soanyways, I'm justgonnaquicklysayherethat I amkindofworkingoffofthistensorflow 2.0, tutorialthatisontensorFlowswebsite.
Now.
I'm kindofgoingtostrayfromitquite a bit, tobehonest, but I'm justusingthedatasetsthattheyhaveand a littlebitofthecodethattheyhaveherebecauseit's a veryniceintroductiontomachinelearningandneuralnetworks.
Butthere's a lotofstuffinherethattheydon't talkabout, andit's notveryindepth.
Sothat's what I'm goingtobeaddinginthereasonwhymaybeyou'd wanttowatchmyversionofthisasopposedtojustreadingthisoffthewebsite.
Sothat's enoughaboutitfeltlike I shouldtellyouguysthatthefirstthingthatwe'regonnabedoingbeforeweactuallystartworkingwithTENSORFLOWisweobviouslyneedtoinstallitnow.
Actually, maybe I'llgrabtheinstallcommandhere, so I havetocopyit.
ButthisistheinstallcommandforTensorflowtwopoint.
Oh, so I'm justgonnacopyhere, Linglebeingdescriptionaswellasonmywebsite, andyoucanseepinkPipinstallhyphenquetensorflowequalsequalstwopoint.
Oh, point.
Oh, hyphen, Alfazero.
I alreadyhavethisinstalled.
I'm gonnagoaheadandhitenteranyways.
Andthehyphen Q I believejustmeansdon't giveanyoutputwhenyou'reinstalling.
Solet's startbyimportingtensorflowgoingfortensorflowas t f likethat.
I don't knowwhyitalwaysshortformswhen I trytodothis.
Butanyways, we'regonnaimportUh, actuallysorryfromtensorflowwillimportcareasnowCaressisan A p I fortensorflow, whichessentiallyjustallowsustowritelesscode.
Itdoes a lotofstuffforus, likeyou'llseeonlysetupthemodelweusecareasandwillbereallyniceandsimpleandjustlike a highlevel a p I That's thewaythattheydescribeit.
Thatmakesthings a loteasierforpeoplelikeusthataregonnabedefiningourown 10 Tsar's andwritingourowncodefromscratch.
Essentially.
Now, anotherthingweneedtoimportisnumbpiesweregonnasay, importif I couldgetthisyearimportsnumbpieasnp.
Andthesepixelvaluesaresimplyrepresentedof, I believe, like, howmuch?
I don't actuallyknowthisscalethatthey'reon, but I thinkit's likeanRGBvalue.
ButinGreecescale, right?
So, forexample, wehave, like, 022 55 where 2 55 isblackandzeroiswhite.
And I'm prettysurethat's howgettingtheinformationinsomeonecancorrectmeif I'm wrong.
But I'm almostcertainthatthat's howthisactuallyworks.
Sothisisgreat.
No, butthisisThesearelargenumbers.
Andremember, I wassayingbeforeinthepreviousvideo, that's typically a goodideatoshrinkourdownitdownsothatit's withwithin a certainrange, thatis a bitsmaller.
Sointhiscase, what I'm actuallygonnadois I'm goingtomodifythisinformation a littlebitsothatweonlyhaveeachvalueoutofone.
Sowe, insteadofhavingNo.
2 55 wehaveanotherone.
Sothewaytodothatistodivideeverysinglepixelvalueby 2 55 nowbecausethesetrainimagesareactuallystoredinwhat's knownas a numbpyre.
Andthat's what I'm hopingThiogivetoyouguysrightnow.
Soifyourememberbeforeweknownowthatareimagesthatcomeinessentiallyaslike 28 by 28 pixels.
Andthewaythatwehavethemiswehaveanarrayandwehaveanotherrainsides, like a twodimensionalrain, haspixelvaluestomakeit 0.10 pointthree, whichisthegrayscalevalue.
Andthisgoesandthere's times 28 Citroofthesethesepixels.
Nowthere's 28 rows, obviouslybecause, well, 28 by 28 pixels.
Sowhatweneedtoactuallydobeforewecaneven, like, starttalkingabouttheneuralnetworkisfigureout a waythatwecanchangethisinformationinto a waythatwecangiveittotheneuralnetwork.
Sowhat I'm actuallygonnadoandwhat I meanmostpeopledoistheydowhat's calledflabinthedata.