Name: トレーニングとテスト 深層強化学習(DQN)エージェント - 強化学習 p.6 (Training & Testing Deep reinforcement learning (DQN) Agent - Reinforcement Learning p.6)
Uploaded: 2021-01-14T10:36:23.000Z
Duration: 48 min 37 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

上級比較級を表す句型

Everybody welcome apart six of the reinforcement learning tutorial Siri's as well as part two of the Deep Que learning as well as de que en and deep cute networks tutorials where we left off.

様態の副詞

We're basically ready to add the train method to her agent and then basically incorporate her agent to the environment and start that literate of training process.

Now and then, I've pulled up the exact versions that I'm using.

Also, people ask a lot like Why am I on paper?

Space is just a nice virtual machines in the cloud kind of cloud desktop, but also high end G pews.

It's great for doing machine learning in the cloud if you want.

I'll put a link in the description referral link.

It wasn't really meant to be a sponsor spot and, like I really do use paper like I use the heck out of paper.

So anyway, and I've got plenty of high end reap use locally, it's just very convenient to use paper space.

But anyways, I've pulled up the exact versions of things I'm using.

These are the exact versions you can feel free to fall along on different version.

It's just if something is going wrong and you want to match my exact versions of things, you can do this.

Also, in case you didn't know what's not all caps, please.

In case, you know, like t install, exact versions would be like Pip three install tensor flow dash GPU double equals 1.13 point 14 examples.

That's how you installed an exact version of something.

So so we're gonna do is come down to our agent class of the D.

And I'm just gonna go to the very bottom here.

I'm gonna make a bunch of space and come up here.

And now what we're gonna do is just add the new train method, so define train, and then we're gonna pass here self terminal underscore states, and then step Whatever.

Okay, So the first thing we want to do is do a quick check to see should we actually train?

So recall we're gonna have this replay memory and then from the replay memory, which should be quite a large memory, we're gonna grab a mini batch that's relatively small, but also is a batch size that is of decent size.

So typically with neural networks like to train in batches of, like, 32 or 64 something like that.

Um, so we're gonna do the same thing here, so we want 32 or 64 to be pretty small compared to the size of our memories.

So in this case, I want to say our max memory size is 50,000 um, and then the least amount that we're willing to train on its 10,000.

And the reason why we want to do that is we don't want to wind up over fitting.

So we want to use this replay memory principle, I guess.

But what we don't want to do is have replay memories so small that every step is trained on the exact same thing.

So we're effectively doing like, you know, hundreds of pox on the exact same data.

So anyway, we're going to say is if Len of self dot replay underscore memory is less than the men replay memory size.

Otherwise, if it's enough, then let's go ahead and get our mini batch, which would be a random dot sample of self doubt.

Replay, memory and the sample size that we want is mini batch underscore size, and we need to do both.

We need to import random and set the mini batch size, so I'm gonna go to the tippy top here.

I'm going to import random, and then I'm gonna come down here.

Mini Batch size will set that to 64 I'll go back to our bottom here.

So now that we've done that, we want to get our accu values.

So don't forget as well that, um, one of the things on my face.

We're actually gonna use some stuff from the other one, too, so just pull it up.

So we actually want to get, um so this bit is kind of handled otherwise both learning rate and then just some of this logic is handled by the neural network.

But we still need the reward discount in that future value.

So to do that, we need to know current que values and future que values.

Okay, so what we're gonna say here is current underscore states is equal to the number pi array.

And there really is a list comprehension here.

We're gonna say transition 04 transition in mini batch.

And then what we want to do here is normalize it.

So again, if you don't know why we're normalizing that check out the basics tutorials.

So any time you have RGB images that it's like 022 55 onward, you can really not have normalized the wrong word.

I've probably missed saying that wrong the whole time anyway, Instead, we're actually trying to do is scale it because all the images are normalized already pretty much, but instead we're actually trying to scale it between zero and one.

That's just the best way for convolution, all neural networks to learn on image data.

You generally want to scale between zero and one or a negative one and one anyways.

So current states, then what we want to grab is the current cues list, and that is also gonna be equal to self dot model dot Predict now pay attention, that SOFA model.

So it's that that model that is the crazy model self top model that predict, uh, and we want to predict on current states.

And then we want to do the same thing with future.

So we're going to say new current states.

This is after we take steps, it's going to be able to the num pyre Ray.

Transition 34 transition in our mini batch, and again we want to do by 2 55 and I'll explain these index values in a second if you don't understand those, or you forgot, uh, should be up here somewhere, um, or is that I think we're gonna have to define that in our environment.

Okay, so now what we're gonna say is future cues.

字幕リスト動画再生

トレーニングとテスト深層強化学習(DQN)エージェント - 強化学習 p.6 (Training & Testing Deep reinforcement learning (DQN) Agent - Reinforcement Learning p.6)

stuff

episode

grab

basically

トレーニングとテスト 深層強化学習(DQN)エージェント - 強化学習 p.6 (Training & Testing Deep reinforcement learning (DQN) Agent - Reinforcement Learning p.6)

stuff

episode

grab

basically

トレーニングとテスト深層強化学習(DQN)エージェント - 強化学習 p.6 (Training & Testing Deep reinforcement learning (DQN) Agent - Reinforcement Learning p.6)