Name: 大規模機械学習 (Large Scale Machine Learning)
Uploaded: 2021-01-14T07:50:07.000Z
Duration: 36 min 10 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

Thank you for being here and participating in this colloquium.

So, I'll tell you about some of the things that are happening in deep learning,

but I only have 30 minutes so I'll be kind of quickly going through some subjects

and some challenges for scaling up deep learning towards AI.

Hopefully you'll have chances to ask me some questions during the panel that follows.

One thing I want to mention is I'm writing a book.

It's called Deep Learning, and you can already download most of the chapters.

These are draft versions of the chapters from my web page.

It's going to be an MIT Press book hopefully next year.

So, what is deep learning and why is everybody excited about it?

First of all, deep learning is just an approach to machine learning.

And what's particular about it, as Terry was saying, it's inspired by brains.

Inspired, we're trying to understand some of the principles, computational

and mathematical principles that could explain the kind of intelligence based

the idea is that these algorithms learn representations.

So, representations is a central concept in deep learning, and, of course,

the idea of learning representations is not new.

It was part of the deal of the original neural nets,

like the Boltzmann machine and the back prop from the '80s.

But what's new here and what happened about ten years ago is a breakthrough that allowed us

to train deeper neural networks, meaning that have multiple levels of representation.

So already I mentioned that there are some theoretical results showing

that you can represent some complicated functions that are the result of the many levels

of compositions efficiently with these deep networks, whereas you might --

or in general, you won't be able to represent these kinds of functions

with a shallow network that doesn't have enough levels.

It means that you're able to represent more abstracts concepts,

and these more abstract concepts allow these machines to generalize better.

So, that's the essence of what's going on here.

So, the breakthrough happened in 2006 where, for the first time,

we were able to train these deeper networks and we used unsupervised learning for that,

but it took a few years before these advances made their way

to industry and to large scale applications.

So, it started around 2010 with speech recognition.

By 2012, if you had an Android phone, like this one, well,

you had neural nets doing speech recognition in them.

For speech, it's changed the field of speech recognition.

Then about two years later, 2012, there was another breakthrough using convolution networks,

which are a particular kind of deep networks that had been around for a long time

of the techniques we discovered along these -- in recent years.

Really allowed us to make big impact in the field of computer vision

So, I'm sure [Faye Faye] will say a few words later about that event and then the role

But what's going on now is that neural nets are going beyond their traditional realm

of perception and people are exploring how to use them for understanding language.

Of course, we haven't yet solved that problem.

This is where a lot of the action is now and, of course,

continues a lot of research and R&D and computer vision.

Now, for example, expanding to video and many other areas.

But I'm particularly interested in the extension of this field in natural language.

You've heard about reinforcement learning.

There is a lot of action there, robotics, control.

So, many areas of AI are now more and more seeing the potential gain coming

So, today, I'm going to go through three of the main challenges that I see

for bringing deep learning, as we know it today, closer to AI.

Of course, for a company like IBM and other companies

that build machines, this is an important challenge.

It's an important challenge because what we've observed is

that the bigger the models we are able to train,

given the amount of data we currently have, the better they are.

So, you know, we just keep building bigger models

and hopefully we're going to continue improving.

Now, that being said, I think it's not going to be enough so there are other challenges.

One of them I mentioned has to do with understanding language.

But understanding language actually requires something more.

So, people are starting to use these recurrent nets you heard about, recurrent networks

that can be very deep, in some sense, when you consider time in order

to combine different pieces of evidence, in order to provide answers to questions.

And essentially, displayed in different forms of reasoning.

So, I'll say a few words about that challenge.

And finally, maybe one of the most important challenges that's maybe more fundamental even is

Up to now, all of the industrial applications of deep learning have exploited supervised learning

where we have labeled the data we've said in that image, it's a cat.

In that image, there's a desk, and so on.

But there's a lot more data we could take advantage of that's unlabeled,

and that's going to be important because all of na information we need to build these AIs has

to come from somewhere, and we need enough data, and most of it is not going to be labeled.

Right. So, as I mentioned, and I guess as my colleague,

Ilya Sutskever from Google keeps saying, bigger is better.

At least up to now, we haven't seen the limitations.

I do believe that there are obstacles, and bigger is not going to be enough.

But clearly, there's an easy path forward with the current algorithms just

字幕リスト動画再生

大規模機械学習 (Large Scale Machine Learning)

sort

essentially

basically

progress