Name: Pythonチュートリアルのディープ強化学習 - ディープラーニングの実装方法を学ぶコース 論文 (Deep Reinforcement Learning in Python Tutorial - A Course on How to Implement Deep Learning Papers)
Uploaded: 2021-01-14T10:23:15.000Z
Duration: 2 h 57 min 11 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

程度の副詞

Today you're gonna learn how to go from a paper to a fully functional implementation of deep deterministic policy.

Grady INTs If you're not familiar with deep deterministic policy greetings or D d.

For short, it is a type of deeper reinforcement learning that is used in environments with continuous action spaces.

You see, most environments have discreet action spaces.

This is the case with, say, the Atari Library say ah, like break out or space invaders, where the agent could move left, right, It could shoot, but it can move left, right and shoot by fixed, discreet intervals.

Fixed amounts, right In other environments like, say, robotics, the robots could move a continuous amount so it can move in anywhere from 0 to 1 minus one plus one.

Anything along a continuous number interval.

And this poses a problem for most deep reinforcement learning methods like sake, you learning, which work spectacularly well in discreet environments but cannot tackle continuous action spaces.

Now, if you don't know what any of this means, don't worry.

I'm gonna give you the rundown here in a second.

But for this set of tutorials, you're gonna need to have installed the opening I Jim, you'll need Python 3.6, and you also need tensorflow and pytorch.

Other packages you'll need include Matt Pot Live to handle the plotting of the learning curve, which will allow us to see the actual learning of the agent as well as numb pie to handle your typical vector operations.

Now, uh, here, I'll give you a quick little rundown of reinforcement learning.

So the basic idea is that we have an agent that interact with some environment and receives a reward.

The rewards kind of take the place of labels and supervised learning, and that they tell the agent what is good.

What is it that it is shooting for in the environment.

And so the agent will attempt to maximize the total rewards over time by solving something known as the bellman equation.

We don't have to worry about the actual mathematics of it, But just so you know, for your future research the algorithm for typically concerned with solving the bellman equation, which tells the agent the expected future returns, assuming to follow something called its policy.

So the policy is the probability that the agent will take a set of actions given it's in some state s, it's basically probability distribution.

Now many types of algorithms such askew learning will attempt to solve the bellman equation by finding what's called the value function.

Function in this case maps the current state and set a possible actions to the expected feature returns the agent expects to receive.

So in other words, Agent says, Hey, I'm in some state meaning some configuration of pixels on the screen in the case off theatre origin Atari Library, for instance, and says, Okay, if I take one or another action, What is the expected future return?

Assuming that I follow my policy actor critic methods are slightly different and that the attempt to learn the policy directly and recall the policy is a probably a distribution that tells the agent what the probability selecting an action is given its in some state s.

So these two algorithms have a number of strength between them.

Grady is a way to marry the strength of these two algorithms into something that does really well for discreet action.

You don't need to know too much more than that.

So in the first video, you're going to get to see how ah, I go ahead and read papers and then implement them on the fly.

Um, and in the second video, you're going to see the implementation of the deterministic policy radiance in pytorch.

In a separate environment, both these environments are in both.

These environments are continuous, and so they will demonstrate the power of the algorithm quite nicely.

You don't need ah particularly powerful GPU, but you do need some kind of deep you to run these as it does take a considerably long time, even on a GPU.

So you will need at least a like, say, a Maxwell Class GP or above.

So something from the 700 Siris on a video side.

Unfortunately, neither of these ah frameworks really work well with a M D cards.

So if you have those, you'd have to figure out some sort of Cluj to get the open sea el implementation to trance compile Tokuda.

Uh, I don't have any information on that.

Grab a snack drink and watch us at your leisure.

I actually did the videos in a separate order.

Reverse order on my channel just so I could get it out.

So I did the implementation and pytorch first and then the video on implementing the paper intensive close second.

But it really is best for a new audience to go from the paper paper video to the pytorch video.

Leave any comments, questions, suggestions, issues down below.

You can check out the code for this on my get hub.

And you can find many more videos like this.

All my YouTube channel machine learning with Phil.

Everybody In today's video, we're gonna go from the paper on deep, deterministic policy ingredients all the way into a functional implementation and tensorflow.

So you're going to see how to go from a paper to a real world implementation.

So the first step in my process really isn't anything special.

Of course, starting with the abstract, the abstract tells you what the paper is about at a high level, it's just kind of an executive.

Summary introduction is where the authors will pay homage to other work in the field.

Kind of set the stage for what is going to be presented in the paper as well is the need for it?

Ah, the background kind of expand on that and you can see here.

It gives us a little bit of mathematical equations and you will get a lot of useful information here.

This won't talk too much about useful nuggets on implementation, but it does set the stage for the mathematics community implementing which is of course critical for any deep burning or, in this case, deep reinforcement learning paper implementation.

The algorithm is really where all the meat of the problem is.

It isn't here and that they lay out the exact steps you need to take two.

So this is the section you want to read most carefully.

And then, of course, they will typically give a table where they outline the actual algorithm.

And oftentimes, if I'm in a hurry, I will just jump to this because I've done this enough times that I can read this.

What is called pseudo code if you're not familiar with that pseudo code is just an English representation of computer code s Oh, we will typically use that we outlined a problem on.

So typically, I'll start here reading it and then work backward by reading through the paper to see what I missed.

But of course it talks about the performance across a whole host of environments.

And of course, all of these have in common that they are continuous control.

So, uh, what that means is that the action space is a vector whose elements can very on a continuous reel number line, instead of having discreet actions of 012345 s o.

That is the really motivation behind deep, deterministic policy grievances that allows us to use deep reinforcement learning to tackle these types of problems, And in today's video, we're gonna go ahead and tackle the I guess pendulum swing up, also called the pendulum problem.

Reason being is that while it would be awesome to start out with something like the bipedal walker, you never want to start out with maximum complexity.

Always want to start out with something very, very small on that scale, your way up, and the reason is that you're gonna make mistakes.

And it's most easy to debug most quick to debug very simple environments that execute very quickly.

So the pendulum problem only has, I think, three elements in its state vector and only a single action.

But either way, it's very small problem relative to something like the bipedal walker or many of the other environment.

You could also use the continues version of the card pole or something like that.

I've just chosen the pendulum for this pretty quickly because we haven't done it before, so it's in here that they give a bunch of plots of all of the ah performance of their algorithm of various sets of constraints placed upon it and different implementations so you can get an idea.

字幕リスト動画再生

Pythonチュートリアルのディープ強化学習 - ディープラーニングの実装方法を学ぶコース論文 (Deep Reinforcement Learning in Python Tutorial - A Course on How to Implement Deep Learning Papers)

process

critical

basically

typically

Pythonチュートリアルのディープ強化学習 - ディープラーニングの実装方法を学ぶコース 論文 (Deep Reinforcement Learning in Python Tutorial - A Course on How to Implement Deep Learning Papers)

process

critical

basically

typically

Pythonチュートリアルのディープ強化学習 - ディープラーニングの実装方法を学ぶコース論文 (Deep Reinforcement Learning in Python Tutorial - A Course on How to Implement Deep Learning Papers)