[MUSIC PLAYING]

[END PLAYBACK]

DAVID J. MALAN: All right.

This is CS50, and this is Yale University.

Welcome to week seven.

So this class marks a transition between the first part of the course, where

we have been learning about the magic wonder world of C--

an extremely powerful low-level programming language that

has allowed us to solve many problems, indeed--

to a second part of the course, where we move

towards more high-level programming languages, such as Python.

In fact, this transition had already begun last week

when we learned about HTML, CSS, and the like.

And we left off, really, by commenting on the fact

that this new programming language, Python, can be conveniently used

to write something like the back end of google.com or facebook.com

or any web service, really, that accepts some parameters,

parse them, possibly look up for some code in a database,

possibly store some code in a database, and it gets back

to the user with dynamic output.

So before we get to that end and to see how Python can used to indeed write

the back end of a web server, it is instructive to see

how Python can be used as a tool, really, to do data analysis,

much like a data scientist will do.

And this is what we are going to see today,

diving into the magic world of machine learning.

But first of all, what is machine learning, really?

So these are a funny sequence of vignettes that I

found online on pythonprogramming.net.

And they represent the stereotypical life of programmers/researchers

in machine learning.

So let's see on the top left here.

Well, society thinks that they are creating hordes of robots,

possibly with the idea to conquer the world.

Right?

Friends think that they're hanging out with robots, really.

Now, parents-- their parents think that programmers in machine learning

spend most of their time in data centers with no windows, apparently.

What about other programmers?

Well, they might think that programmers do fancy mathematics.

And what about themself?

Well, they typically think that they're involved with some fancy visualization,

data analysis.

But at the end of the day, what they really do-- and here we are--

is using Python, and not just using Python to implement some algorithm,

but really using Python to import some algorithm as we are seeing here.

So we do not know Python yet.

But this line of code looks extremely readable, isn't it?

There is English there.

It says "from sklearn"-- we don't know what this is yet-- "import"--

I already mentioned svm, support vector machine.

It's a function to run a machine-learning algorithm.

So we don't know Python yet, but we're already

able to decipher, more or less, what is going on.

And indeed, sklearn, as we will see, is a so-called module

in Python-- a library, if you wish, in C-- from which we

are importing an algorithm, a function.

Now, this line exemplifies a characteristic feature of Python--

namely, it readability.

And it is often the case that Python code is referred to

as being similar to pseudocode, precisely for the fact

that it allows us to express very powerful ideas with a couple of lines

that are extremely readable.

Now, this very characteristic of Python, together

with our familiarity, at this point in the course, with C,

is what will allow us today to quickly see

how Python can be used as a data processing tool

to implement and run machine learning algorithms.

Well, so back to the questionn-- what is machine learning, really?

So as the previous sequence of vignettes were suggesting,

in the popular culture, at least, machine learning

is often associated with the world of AI, artificial intelligence.

Typically, we think of machine in terms of robots.

And literally, there are countless science fiction movies

about this theme, typically representing some robots turning evil

against humanity or in this line.

Indeed, this is a modern interpretation of an old fear

dating back, possibly, to Frankenstein in the 1800s and beyond.

But really, if we think of the way machine

learning is having an impact on our lives on a day-to-day basis,

it's not necessarily related to robots, per se.

It has more something to do with the following set of applications

that we indeed use on a daily basis.

So lets just think of search engines, for instance,

that allow us to look for whatever we might feel like in the world wide web,

and in a matter of literally milliseconds to get back

an order, at least, of results based on the query that we enter.

Think about image recognition, the possibility

to catalog, to search for an image based on its subject.

Think about speech recognition software.

These days, we can all talk to our phone and ask, what's the weather like

or show me the movies around me.

Or finally, just for these four sets of application

I mentioned, the world of natural language processing,

the amazing ability to translate a document of text from one language

to another in real time or the ability to, say, infer

the meaning, the semantics of a document with no human intervention.

So these are just a few of the applications that are indeed backed up

by machine learning algorithms.

And here by the word machine, we really mean an algorithm running, most likely,

on the cloud in a data center.

And today, we use these applications on a daily basis.

And so have you ever wondered what they're all about?

How can we get started to thinking to design one of these applications?

And this is what we're going to learn today.

So in particular, we will be focusing on two applications-- image recognition

and natural language processing.

But before that, let's go back to week zero.

So we have seen this diagram early on in the course.

It represents a general-purpose algorithm.

The black box is an algorithm that takes some input from the left-hand side.

It processes them, and it delivers the user with an output.

Now the class of application that we're going to see today

very much fits into this framework.

Just think about image recognition, for instance.

Well, we might want to have an algorithm that

takes as inputs images-- say here an image of a horse, image of a car--

and is capable or realizing or recognizing that there is, indeed,

a horse or a car in that image and get back to us

with strings, such as "this is a horse."

Or, well, in the world of natural language processing,

think about an algorithm where we could do something of the following--

say I want to pass as an input to this algorithm one of my favorite novels,

1984 by George Orwell.

So this is just a part of it-- "BIG BROTHER is watching you."

But say that we want really the entire book to be fed up as an input

to this algorithm.

We want the algorithm to be able to recognize the semantics of the book.

So we want the algorithm, with no human intervention,

to be able to get back to us and tell us, look, Patrick,

this book is about politics.

It's about propaganda.

It's about privacy.

So in both of these two applications-- image recognition and natural language

processing-- it is already clear that what the machine learning

algorithm is doing is trying to infer some hidden structure in the input,

right?

And if we phrase the problem like that, well, we have seen something like that

earlier on in the course-- in fact, many examples of a scenario

where we are given an input with some hidden structure,

and we want to design-- you have done that in problem set four--

an algorithm that can decipher, that can crack the input and gives us an output.

So this is Whodunit.

And per the specific, we were given an image with some red noise.

And you were told to design an algorithm to get back with the hidden message.

So in this respect, you might think applications such as image recognition

seem to share the similar behavior, isn't it?

But there is a key difference.

Anyone can spot the difference?

We're getting started.

So one difference in the picture is that instead of just one image,

there are two images.

And indeed, that is somehow to the point in the sense

that with problem set four, we were given an image with a hidden message,

indeed.

But we were told the structure of the hidden message.

In other words, we were told that the image had a lot of red noise,

and we were told that in order to decipher, to really find

the hidden message, we should lower the intensity of the red pixel

and possibly rise the intensity of the other pixels.

But in machine learning applications, this

is not typically what we have in mind.

So we want an algorithm that can work with any sort of image

we may want to feed it with.

And this is what is bringing us to one of the key differences

in the class of application that we are going to see,

namely the fact that it is the algorithm itself

that is trying to find out the hidden structure in the input.

And the way you can do this is by having access to what is called training data.

In other words, even if we want to design an algorithm that

is capable to tell us, Patrick, look, this is a horse, in order to work,

this algorithm needs to have access to a lot of images of horses.

And based on these images, it is able to figure out the hidden structure

so that once you feed, again, an image of a horse, it can say,

this is a horse.

And this is what we are going to see today,

starting precisely from the example of image classification.

So what is the example we're going to play with?

It's the following-- suppose that we are given a data set of handwritten digits.

So there are 10 digits from zero to nine.

And for each of these digits, we are given

a collection of handwritten images representing that digit.

So for instance, for digit zero, we are given many of these images.

And each of is having a different way of writing a digit zero.

So what we set as our goal for today is to actually design, in this case,

an algorithm that can take an image of digit zero

or an image of digit six or any other digit

and, indeed, get us back with the string "0" or "6" or so on.

And the way you can do that, as we will see,

is by having access to the so-called training

data that is inferring the [INAUDIBLE] structure behind the images

we want to use.

But before we get to talk about images, let's us just abstract a little bit.

And let us think that we live in a one-dimensional world, a line land,

if you wish, where the only thing that we can see is a straight line.

Say we go to school, and the first thing that we are told in day one of school

is that, OK, I see a point here.

The teacher is telling us, well, look, Patrick.

This is number zero.

Then we go again, day two of school.

We see another point.

The teacher is telling us, this represents a zero.

Day three, we see another point.

It's the only thing that we can see in a one-dimensional world.

And the teacher is telling us, well, this is a six.

And so on.

So day four, we see another point.

This represents number zero.

And so on.

So this is representing number six.

This will represent the number six.

So in other words, we have been exposed to a labeled training

set of data-- points and the associated label corresponding

to the digits they represent.

So say now that we are presented, the next day of school,

with a so-called test point over here.

And we are asked, what should this number be?

What should this point represent?

Which digit?

Anyone?