Name: 生成モデルの基礎 - 型破りなニューラルネットワーク p.1 (Generative Model Basics - Unconventional Neural Networks p.1)
Uploaded: 2021-01-14T10:44:09.000Z
Duration: 14 min 12 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

現在進行形

what's going on, everybody and welcome to a new Siris on hacking around with neural networks.

Just kind of doing unconventional things with neural networks.

未来形

And the type of model that we're going to start off with is the generative model.

So what I have here is a simple program, basically where I just asked the model, Hey, draw me a number of zero through nine.

I'll pick the number I'll say, Please draw me a five, for example, and that five becomes a sort of primer for the generative model.

The gender of models then generates me something like a five now.

The interesting thing about this model, though, is that first of all, this five is a totally unique new novel.

And if I ask you to draw me yet another five, what it's gonna do is draw me yet another unique new novel.

Five and so So the reason why I think generative models air interesting is one that they're still like a really like area of research and a lot of people in the fields of deep learning, Um, kind of write off generative models.

I'm drawing a four, by the way, Just so you know, four.

Anyways, the people in the fields of machine learning tend to write off generative models because they don't quite fit or paradigms today of problems that we'd like to see solved.

So classifier is, on the other hand, with deep learning.

There's kind of two ways that class fires with deep learning have improved, and that's with incrementally better accuracy and results.

And then also with, um, there's our nice beautiful for you get the idea, um, and then also with solving problems where there's, like, millions of samples that we just could not train with support vector machines.

But arguably, I would say those results are kind of based more on, ah, just using GPS generally So g PG BYU's, um and so those two things kind of happened at the same time.

So anyways, so incrementally better results and then able to solve some classification problems that we've never had before.

But with generative models there, it just opens up on entire new door.

And so just cause we can't think of, uh, use cases for them, doesn't mean, we need to write them off immediately.

Um, I think probably part of the problem is a lot of times funding for research and stuff is done by companies and stuff like that that have a use case in mind.

And I think that's why we see these as being useless.

Besides, you guys so python programming, and that's our support.

And, uh so anyways, we get to play around and do fun things.

So with that, let's get into the tutorial where we're going, t get to this point, but also do some other playing around with other things, Um, and just kind of poke around.

I'm working with his python 3.6 tensorflow 1.7, and then we're also going to at least start off using the following package.

So, um, the the commit for this exactly is for one E b f D.

Um, it doesn't look like it's to actively maintained, so maybe it won't be a problem, But if anything is not working, you can get the same python version, same tensorflow and same same package that we're gonna use here.

So go ahead and clone, download that and somehow got a copy of it.

And then what we're gonna do is we'll break out of this one, move that aside, and once you have the package will go ahead and extract it.

So this is a character level generative model, and what's gonna do is you just give it any characters and then what it's gonna do is you're gonna say OK, with all these characters, I want you to you train on it and then be able to generate me something similar to what you've seen.

So it's It's like a classifier in the sense that you can produce.

Um, you know, it's there, there's an input and there's an output, and the goal is to fit the output to the input based on the training data.

So it's very similar to class fires in that sense, but the difference is a generative model can take a variable number of input, and it can produce a variable sized output.

And with tensorflow recurrent, we do have these dynamic, recurrent layers which are slowly becoming capable of doing this sort of thing.

So I think the best way Thio to show you guys what generative models do is to just kind of use one and work with it.

So So this is the package we can check out.

But basically, you're gonna pass in, You know, you're an astronaut pie.

It's gonna look in your data dirt for something, But we can We can use the command line arguments, Thio pass some other form of data.

We're gonna ask it to you know, we were gonna tell it where to save, where to put the tents are bored logs basically the size of the network, all that stuff.

But there's there's defaults for all of this.

Um, so you can kind of look through that once you start getting past sequence length, maybe number of at a pox.

Um, you You probably are are done changing things.

I wouldn't suggest you mess with these until wait on one.

But the main thing is to look out for or the size of the model, um, maybe batch size.

If you make a model too big that it can't fit in the memory.

If it's not too big, you could increase batch size to use more of your utilize more of your jeep.

You possibly, um, what was the other thing?

Right now, it's gonna look at the last 50 characters each time it wants to generate.

One new character is gonna look back 50 characters.

Also, we can see that there's maybe Cem Cem starting data.

So let's check out data slash tiny Shakespeare.

And basically, we're looking for input dot text, so it's good.

Let's go data Tiny Shakespeare input dot text And here's what we have.

So as we can see, this is some Shakespeare.

So it's a play, basically, and and there's a few things here that you want to kind of pick up.

One is, you see, there is some structure to it.

You've got basically what appears to be the structure of name colon.

New line sentence, New line, New line name, colon, New line sentence, New line, New line.

Um so we have that also we can see that you basically got name, which should just be, like one or two words.

Most likely Ah, then the sentences could be highly variable.

Um, and then it's like old English, basically.

Okay, Um, so yes, So that's our training set.

The other thing that I really think it's cool about generative models is the size of data that's required.

So So the fact that if it works spoiler alert, it works.

If you haven't seen them spoilers already.

Um, that's really cool that you can do this with such a small amount of data.

So anyways, so that's our starting data set.

So coming back into this main door where we've got trained up pie, let's go ahead and actually just run python trained up high.

But slowly entering into the world of generative models, um, let's make sure this actually starts training.

And once it does, I'll just posit, um and then all of a NPAs.

Um, you're probably a CPU or maybe a lesser GPU.

But I'm pretty sure that, uh, this doesn't even come close to fully utilizing my gp.

I could see my mouth's moving pretty pretty smoothly.

Anyways, um, what I'm gonna do is just pause it recording while this is running.