Name: ディープラーニングによるより複雑な数学 - 型破りなニューラルネットワーク p.11 (More Complex Math with Deep Learning - Unconventional Neural Networks p.11)
Uploaded: 2021-01-14T10:36:21.000Z
Duration: 24 min 43 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

現在進行形

命令形

And welcome back to the unconventional neural network Siris In this video, only doing is running over some of the results from the neural network that was trained just to do addition, we're gonna talk about some statistics.

We're gonna see how it did and then what we're going to be looking looking to do, moving forward.

So one thing I did was I wrote a few different kinds scripts just for comparing how we're doing as time went on.

So so one thing week we can do is inside model.

We should have, um, the train log directory.

So one option we have is to just do si m d Here.

So we can We can obviously see how loss was doing.

So actually, this model could have continued to train.

Well, actually, I'm sorry this problem isn't fully loaded yet.

Yes, you can see it's slowly certain level out.

Need to wait for that to to finish processing.

So when one way that we can see how models doing is to look at train log butt, part of the issue is with with loss.

The loss is really all about the weights.

And when when we're looking at something like a mathematic problem, or in our case, like when we're doing the Mt.

Chatbots, even there's a reason why we used other metrics like Blue and PPL because, uh, loss could mean, Sure, we're learning something.

But are we learning the thing we were hoping to learn?

That might be a different question, so you often want to come up with separate metric for that.

So what I did was just manually as this model trained, I just took the output Dev as they came through, and I just rename them quickly.

So as they were coming, um, I would just say at what step point was that depth.

So at five k steps, 10 case steps and and so on all the way to 57 k when it finished training um, after about 100 K, I just went to bed.

So that's why I didn't I didn't get anything in between there.

Hopefully, our tensor board is is done now, so we can kind of pull that up, and I'm just gonna refresh it.

Okay, so So we can actually see after about 1 20 k steps.

It looks like I'm gonna guess that the learning rate was taken down there as well, Let's see, at 120 0 actually, yes.

So right before I went to bed, Um, I recognize that the learning rate was one negative five or should have been wanting negative five.

I wonder why That says one you negative for, um weird.

One negative for So the middle ground here is 20 negative five.

Anyway, um, before I went to bed, that was gonna be the last epoch, and I thought it was still only about 50% accurate.

So interestingly enough, we were actually kind of done learning.

Um then I allowed it to jump back up and look how we just boom just jumped away down in los Um so if anything looking back, this is actually the first time I'm looking at this, uh, that jump up in learning rate.

Basically, what that tells me is we jumped down way too early.

We shouldn't have jumped down that early because putting it back up like we were were fixing a plateau right there basically, but allowing it to jump back up again comes back down here, and that's probably what sealed the deal.

I'll keep that in mind because we're gonna make things a little more complex here, So, uh, cool.

But that doesn't tell us, You know, learning rate doesn't tell us.

So instead, what I used was the output deva file.

Ah, do sublime text and we can see Okay, that's basically the output from the, um, the model.

And it's because it's a character level model.

The spaces aren't actually spaces spaces between three outputs.

Now, the input to these, um, was basically would be like new data on then from.

So yeah, from here, we should get this obviously minus the spaces.

And the question is, is this corrector is incorrect now, even at 5000 steps?

Um, I just I know that this is probably about Kurt lips.

I know it's about correct because they're pretty.

They're at least close there within the same range.

So what I did was I just kind of manually saved those output Debs as they go.

So those were just that's, like, our way of validating things.

So every 5000 steps would about 5000 steps.

It would go to the thes testing one's, which contains samples out of So they're out of sample testing from, like, the training data itself.

So this should be samples that the model has never seen, and this is how it does on those.

So what I did was at least started with this file.

So let me open that up and we'll talk about that.

And what it's gonna do is gonna open up the test file.

So test from it's gonna open up the output file, and that's gonna open up test dot too.

And then what we're gonna do is just create a couple of counters here.

And you're Anyway, um, we're gonna do is we're gonna count, basically, how often were we correct versus how many total samples did we have in the set?

And, uh, and then we can get an actual accuracy rating, so that was like, the first thing I started doing.

字幕リスト動画再生

ディープラーニングによるより複雑な数学 - 型破りなニューラルネットワーク p.11 (More Complex Math with Deep Learning - Unconventional Neural Networks p.11)

basically

negative

accurate

force