LIVE-CODE TRAINING EPISODE, OR EVENT, AND IT IS SPONSORED BY

SPELL. YOU CAN GO TO SPELL.COM/CODING

SPELL.COM/CODINGTRAIN. THERE'S AN INTRO TO SPELL PLATFORM.

WE HAD A TUTORIAL ON HOW TO

TRAIN A STYLE MODEL. WE

HAVE NABIL HASSEIN WHO IS GOING TO TALK ABOUT HOW TO

THANE TRAIN AN LSTM MODEL. THANK YOU TO WHITE COAT

CAPTIONING, THERE SHOULD BE CAPTIONS SAYING THINGS LIKE

MANGO, BLUEBERRY, BANANA, HOPEFULLY. AND I WILL TELL YOU

ABOUT NABIL, HE IS A BROOKLYN-BASED ARTIST, WEB

DEVELOPER, AND ALUMNI OF

NYU. HE .

YOU CAN FIND OUT MORE ABOUT NABIL AND THE WORK HE IS DOING

AND THE PROJECTS HE IS ININVOLVED IN AT THE WEBSITE.

AND NOW, RIGHT NOW, THE DESCRIPTION IS PRETTY BARE, IT

IS NABIL'S BILLOW AND A FEW WORDS ABOUT WHAT WE'RE GOING

TO DO TODAY. SO YOU SHOULD LOOK AT THE LIVE CHAT HERE TO MAKE

SURE THAT EVERYTHING IS WORKING. I WILL BRING NABIL ON AND LOOK

AT THE LIVE CHAT AND LET ME KNOW IF THERE ARE TECHNICAL ISSUES WE

CAN FIX BEFORE WE GET STARTED WITH THE TUTORIAL. ANYTHING I

MISSED? SO I WILL BRING NABIL IN, AND

WE'RE GOING TO AWKWARDLY SWITCH PLACES, I WILL WATCH THE CHAT,

IF THERE ARE QUESTIONS, POST THEM THERE.

OKAY, THANKS. AND HERE WE GO. WELCOME, NABIL.

NABIL HASSEIN: WELCOME, EVERYONE. THANK YOU FOR THE

GREAT INTRODUCTION, AND THANKS SPELL FOR PAYING ME TO MAKE THIS

VIDEO, OR DO THE LIVE STREAM. I HAVE AN OUTLINE OF WHAT I

PLANNED TO GO THROUGH. SO I GUESS I WILL INTRODUCE MYSELF.

I'M NABIL, I LIVE IN BROOKLYN, I'M A FREELANCE TECHNOLOGIST,

EDUCATOR, I DO SOME OTHER THINGS. AGAIN, THANK YOU SPELL

FOR SPONSORING THIS VIDEO. SO THIS LIVE STREAM IS ABOUT HOW

TO TRAIN AN LSTM MODEL USING THE SPELL PLATFORM, SO ON SOME

REMOTE MACHINES SOMEWHERE. AND THEN HOW TO USE THAT MODEL

THAT WE'VE TRAINED USING A LIBRARY CALLED ML5.JS, A

BROWSER-BASED FRONT END LIBRARY FOR USING MACHINE LEARNING

MODELS. SO IN THIS VIDEO, I WILL TRY TO DO A FEW THINGS, TRULY

LIVE FOR YOU HERE TODAY. I'M GOING TO EXTEND A PROJECT THAT I

DID AT THE SCHOOL FOR COMPUTATION, WHICH DAN

MENTIONED, LAST SUMMER. SO THE PROJECT, I CAN SHOW YOU THE

VERSION OF IT THAT IS LIVE ON THE WEB.

SO THIS I TRAINED USING A MARKOV MODEL. SO THIS IS A RHYME

GENERATOR, BASED ON THE LYRICS OF ONE OF MY FAVORITE EMCEES, MF

DOOM. EVERY TIME I CLICK A RHYME BUTTON, IT GENERATES

RANDOM RHYMES PATTERNED ON HIS LYRICS. THIS VIDEO ISN'T REALLY

ABOUT THAT PROJECT, IT IS ABOUT HOW TO USE SPELL. BUT IT GIVE

YOU A LITTLE BIT OF BACKGROUND, AND THE CODE IS LIVE IF YOU WANT

TO -- OR THE CODE IS PUBLIC ON GITHUB IF YOU WANT TO CHECK IT

OUT AT SOME POINT. AND I USED A LIBRARY CALLED

PRONOUNCING, WHICH WAS CREATED -- AND I'M NOT SURE IF TOO MANY

PEOPLE -- A LOT OF WORK WAS DONE BY ALLISON PERISH AND I USED

THAT LIBRARY TO GENERATE A PAIR OF RHYMING WORDS, I TRAINED THE

MODEL BACKWARDS ON THE INPUT TEXT SO I CAN GENERATE THE REST

OF THE SENTENCE GOING FROM THE END TO THE BEGINNING, AND THEN

REVERSE ORDER. DANIEL SHIFFMAN: APPARENTLY

THERE IS STATIC IN THE AUDIO. LET ME SEE IF I CAN HEAR IT

HERE. NABIL HASSEIN TEST : TESTING,

TESTING. SHOULD I MOVE THE MIC? UNCLIP IT FOR

A SECOND.

NABIL HASSEIN: OKAY, SO TALKING ABOUT THE VUD VIDEO, TRAINING AN

LSTM MODEL. SO LET'S GO AHEAD AND GET INTO IT.

SO THE NEXT THING: SO I'M NOT REALLY GOING TO TALK TOO MUCH IN

THIS VIDEO ABOUT THE THEORY OF NEURAL NETWORKS OR WHAT AN LSTM

MODEL IS, BUT IT STANDS FOR LONG-SHORT TERM MEMORY. IT IS A

SPECIFIC TYPE OF RECURRENT NEURAL NETWORK AND WHAT IS

USEFUL ABOUT RECURRENT NEURAL NETWORKS IS THE WAY THEIR

ARCHITECTURE INCLUDES LOOPS. AND THAT CAN BE USEFUL FOR KIND

OF KEEPING DATA AROUND IN THE NETWORKS, SO TO SPEAK, WHICH IS

USEFUL FOR APPLICATIONS INVOLVING NATURAL LANGUAGE,

HUMAN LANGUAGE. BECAUSE CONTEXT MATTERS SO MUCH

IN LANGUAGE. LIKE, PREDICTING THE NEXT

CHARACTER, OR THE NEXT WORD, MIGHT BE -- YOU MIGHT GET A MUCH

BETTER PREDICTION IF YOU ACTUALLY REMEMBER WHAT WAS SAID,

EVEN SOME WHILE AGO, MAYBE MUCH EARLIER IN A LONG SENTENCE. SO

I HAVE A FEW QUICK REFERENCES HERE, WHICH RIGHT NOW ARE A

LITTLE OLD. BUT THIS IS -- THESE ARE WHAT I READ TO LEARN A

LITTLE BIT MORE ABOUT RECURRENT NEURAL NETWORKS. SO THERE'S A

BLOG POST, THE UNREASONABLE EFFECTIVE OVER RECURRENT NEURAL

NETWORKS AND THERE'S A BLOG POST CALLED UNDERSTANDING LSTMS, OR

-- DID I TYPE ALL OF THIS SOMEHOW? I MUST HAVE. I WILL

FIX THAT. YEAH, IT IS CALLED UNDERSTANDING

LSTMS. SO, YEAH, THIS GIVES A LITTLE

BIT OF OVERVIEW OF THE STUFF I WAS TALKING ABOUT. HUMANS ARE

THINKING FROM SCRATCH EVERY SECOND, YOU UNDERSTAND A WORD

BASED ON THE UNDERSTANDING FROM PREVIOUS WORDS, AND WITH THE

NETWORK, THAT'S WHY WE ARE USING THIS LSTM MODEL.

AND, YEAH, I KNOW THAT BEFORE I HAD THE CHANCE -- WHILE

PREPARING FOR THIS VIDEO TO WATCH A VIDEO ON AN OVERVIEW OF

THE SPELL PLATFORM, A LINK TO THAT WILL BE ADDED TO THE VIDEO

DESCRIPTION AND YOU CAN GET MORE IN-DEPTH ABOUT USING SPELL, AND

I WILL MENTION SOME THINGS ABOUT USING SPELL AS WE GO THROUGH

THIS. SO WHEN YOU WANT TO DO A PROJECT LIKE THIS, FIRST YOU

HAVE TO GET YOUR CORPUS OF DATA. AND SO IN THIS CASE, SINCE I WAS

GETTING SONG LYRICS, I USED A SITE CALLED GENIUS.COM, WHICH

YOU MAY BE FAMILIAR WITH, IT A POPULAR LYRICS WEBSITE. AND THE

MAIN THING I USE IT FOR IS READING LYRICS.

AND SO WHAT I'M GOING TO DO, I WILL DO EVERYTHING FROM SCRATCH,

SO TO SPEAK, SO YOU SHOULD BE ABLE TO FOLLOW ALONG IN THEORY.

SO THIS IS OVER A FOLDER THAT I USED TO PREPARE. WHAT I WILL DO

IS MAKE A NEW FOLDER, CALLED SPELL LIVESTREAM, I WILL DO

EVERYTHING FROM INSIDE OF THIS FOLDER, WHICH IS LIVED SOMEWHERE

IN MY COMPUTER. SO RIGHT NOW, THIS FOLDER IS EMPTY.

AND SO THE FIRST THING THAT I'M GOING TO DO IS CLONE MY

GENERATIVE DOOM REPOSITORY FROM GITHUB. AND THERE IS ONLY ONE

FILE IN THERE THAT I CARE ABOUT, SO MAYBE -- MAYBE I WILL NOT

CLONE THE WHOLE FILE STORY. I WILL GET THE ONE FILE. SO I'M

JUST GOING TO -- I DON'T KNOW WHERE THIS IS.

DID I PUSH IT UP? I HAVE SO MANY BRANCHES HERE. I WILL USE THE

ONE THAT I HAVE ON MY COMPUTER. SO I'M JUST GOING TO COPY A FILE

THAT I HAVE ON MY COMPUTER INTO THIS FOLDER. SO WHERE WAS THAT,

IN SPELL

SPELLDEMO/GENERATIVEDOOM/DATA. I HAVE A FILE, INPUT.TXT THAT I

BROUGHT A COPY OF INTO THE CURRENT DIRECTORY. SO YOU CAN

SEE THIS IS

JUST A LIST OF LYRICS, THIS IS MY CORPUS. AND IT IS WORTH

NOTING THAT THE DATA SET I'M USING FOR THIS EXAMPLE ISN'T

THAT BIG. WE CAN CHECK THE SIZE OF IT WITH THE COMMAND LINE

UTILITY, DU, FOR DISK USAGE, AND YOU CAN TELL HOW BIG THIS FILE

IS. IT IS ABOUT 308 KILOBYTES. IT IS NOT HUGE.

NORMALLY WHEN YOU ARE TRAINING MACHINE LEARNING MODELS, THE

MORE DATA, THE BETTER. BUT I GOT ALL THE LYRICS I COULD FIND.

THIS IS REALLY THE MOST THAT I COULD GET. SO THAT'S WHAT WE'RE

GOING TO USE. COOL.

SO IT IS ALSO WORTH NOTING THAT YOU HAVE TO CLEAN THE DATA

BEFORE YOU TRAIN IT. SO I CAN ACTUALLY GO AHEAD AND SHOW -- I

CAN GO AHEAD AND SHOW THE CODE THAT I USED TO GET THE LYRICS.

IT IS ON MY GITHUB IF YOU WANT TO CHECK IT OUT.

SO LET'S PUT IT

OVER HERE. SO I HAPPENED TO DO MY SCRAPING

USING PYTHON, YOU CAN USE ANY LANGUAGE, NODE.JS, RUBY, OR

WHATEVER YOUR LANGUAGE IS. I USED A PYTHON LIBRARY, BEAUTIFUL

SOUP, WHICH IS VERY USEFUL FOR WEB SCRAPING. AND SO IT SO

HAPPENS THAT GENIUS.COM HAPPENS TO KEEP THEIR LYRICS IN, LIKE,

THEIR URLS, THEY FOLLOW A PATTERN LIKE THIS,

GENIUS.COM/ARTIST NAME AND THE NAME OF THE SONG, AND THEN I

USED ANOTHER PYTHON LIBRARY, REQUESTS, TO JUST GO AHEAD AND

FETCH ALL OF THESE DIFFERENT THINGS. SO THIS IS THE BASIC

IDEA, I'M NOT GOING TO GO INTO FULL DEPTH, BUT I HARD-CODED A

LOT OF NAMES OF DIFFERENT SONGS INTO HERE AND THEN I HAVE A MAIN

LOOP WHICH LOOPS THROUGH EACH ARTIST'S NAME, BECAUSE DOOM HAS

RECORDED UNDER DIFFERENT NAMES. AND THEN THE SAME THING FOR THE

ALBUMS, AND THEN THE SONGS, IN ORDER TO GO AHEAD AND FETCH ALL

OF THIS

DATA. WHEN YOU GO DIRECTLY TO SOME,

YOU KNOW, LYRIC'S WEBSITE, WHEN YOU FETCH THE DATA ON THE PAGE,

YOU GET A LOT OF OTHER DATA YOU DON'T CARE ABOUT IN THE HTML.

SO YOU HAVE -- AN IMPORTANT STEP IS TO CLEAN THE DATA, SO WHEN

YOU ARE TRAINING THE MODEL, YOU ARE ONLY TRAINING IT ON THE

CORPUS THAT YOU CARE ABOUT, YOU ARE NOT TRAINING IT ON THE

BRACKETS OF THE HTML TAGS THAT YOU DON'T ACTUALLY WANT.

SO I THINK I HAVE THE CODE THAT I USED TO CLEAN IT ON GITHUB,

AND IF NOT, THERE ARE OTHER RESOURCES YOU CAN USE TO LEARN

ABOUT DATA CLEANING. THIS IS ABOUT TRAINING MACHINE LEARNING

MODELS USING SPELL AND USING THEM IN THE BROWSER. LET'S GO

BACK TO THAT. AND PROJECT GUTENBERG, THAT'S A RESOURCE

WITH A LOT OF FREE TEXT IN THE PUBLIC DOMAIN THAT YOU CAN JUST

USE. AND WEB SCRAPING WITH NODE.JS IS ANOTHER RESOURCE FOR

DOING THIS KIND OF THING. AND ALTHOUGH MY SCRAPE.PY FILE IN

THE DOOM REPOSITORY DOESN'T SHOW THIS, THE ORIGINAL VERSION, I

JUST KIND OF KEPT EACH FILE -- I KEPT EACH SONG IN ITS OWN FILE

OF LYRICS. BUT THE MACHINE, WHAT I'M GOING TO SHOW YOU NEXT,

WORKS WITH AN INPUT THAT'S ONE BIG INPUT FILE, INPUT.TXT AND I

DID BORING SHELL SCRIPT STUFF THAT CONCATENATES THE FILE

TOGETHER. AND THAT'S ALL I WANTED TO SAY ABOUT GETTING

DATA. SO LET'S GET INTO THE MAIN THING. SO I DID THIS PART,

I CREATED A NEW DIRECTORY FOR ALL OF THIS STUFF TO

LIVE IN. SO IT IS A GOOD HABIT TO USE

VIRTUAL N, AT LEAST WITH PYTHON DO, I UNDERSTAND THAT THINGS

HAVE MOVED ON A BIT WITH PYTHON THREE, BUT I'M ON THE PYTHON

TWO. SO I'M GOING TO USE

THIS VIRTWOMAN UAL N THING TO KEEP MY DEPENDENCIES ISOLATED.

AND I HAVE SOME OTHER VIRTUAL ENDS ACTIVE, I SEE IT FROM MY

COMMAND PROMPT OVER HERE. I RAN THE COMMAND, DEACTIVATE, TO GET

RID OF THAT. I WILL CLEAR THE SCREEN, TO MAKE IT LESS NOISY

HERE. AND AND THEN I WILL CREATE A NEW VIRTUAL N, SPELL

VIDEO

VIRTUAL N. SO IT IS SETTING UP A NEW

VIRTUAL ENVIRONMENT WHICH LIVES INSIDE

OF THIS FOLDER HERE. AND THE WAY YOU USE PYTHON VIRTUAL END

TO GET IT ACTIVE, SPELL,

VIDEO VIRTUALENV, SOURCE, AND THEN THE PATH TO

ACTIVATE SCRIPT. AND NOW YOU CAN SEE MY PROMPT

CHANGED BECAUSE I HAD MY TERMINAL PREFERENCES SET UP THAT

WAY, I CAN REMEMBER WHICH VIRTUALENV I'M IN.

OKAY, SO I DID THAT. I GOT THE INPUT FILE, WHICH I SHOULD PUSH

IT UP. I HAVEN'T ACTUALLY PUSHED UP TO THE GITHUB, LIKE

THE ONE FILE VERSION, BUT LIFE IS MESSY.

SO I WILL GET THE PROPER VERSION FROM GITHUB IS THIS REPOSITORY

CALLED TRAINING LSTM. SO I'M GOING TO CLONE THAT. AND WE

WILL GO AND TAKE A LOOK AT THAT REPOSITORY AND ITS READ

ME. . I WILL CLONE, COPY, AND PASTE

IT IN. SO YOU CAN SEE THAT THIS -- FROM

THE DESCRIPTION OF THIS REPOSITORY, TRAINING IN LSTM

NETWORK AND USING THE NETWORK ML5.JS IS HIGHLY RELEVANT TO

WHAT WE ARE DOING IN THIS VIDEO. SO THIS REPOSITORY'S READ ME ARE

BASED ON WHAT YOU WOULD WANT TO DO IF YOU WERE TRAINING THE

MACHINE LEARNING MODEL LOCALLY ON YOUR OWN COMPUTER.

AND SO IF THAT'S WHAT YOU WANT TO DO, YOU CAN GO AHEAD AND

FOLLOW THIS. SINCE THIS VIDEO IS ABOUT HOW TO USE SPELL,

THAT'S WHAT CAN ARE I WILL DO. I WILL NOT FOLLOW THE DIRECTIONS

EXACTLY, BUT CERTAIN PARTS OF IT. SO I CLONED THIS

REPOSITORY, THAT'S THE LAST COMMAND THAT WHERE RAN. I RAN.

I WILL ENTER INTO THAT REPOSITORY. AND THEN WHAT I

WANT TO DO IS CREATE A DIRECTORY INSIDE OF HERE CALLED DATA. AND

THENP I'M GOING TO GO AHEAD AND MOVE AHEAD THAT INPUT.TXT FILE

INTO THAT DATA DIRECTORY. OR A COPY OF IT, RATHER. I

COULD DELETE AND GET A DIRECTORY, BUT WHATEVER.

OKAY, GREAT. SO THIS IS THE SET-UP.

WE HAVE A REPOSITORY, WE HAVE THIS REPOSITORY LOCALLY THAT IS

GOING TO HELP US TRAIN THE LSTM NETWORK USING TENSORFLOW. AFTER

WE TRAINED THE MODEL, WE CAN USE IT IN ML5.JS. SO WE'RE DONE

WITH THE SET UP, SO LET'S GET INTO TRAINING THE MODEL.

SO AGAIN, THIS IS THE LINK THAT YOU CAN USE TO SIGN UP FOR SPELL

IF YOU HAVEN'T ALREADY. IT SO HAPPENS THAT I HAVE ALREADY.

SO I SHOULD BE LOGGED IN HERE, I WILL MAKE SURE I DON'T HAVE TO