Everybody.

And welcome to part seven of the A I in Starcraft two with Python Siris in this video in the coming videos, we're going to be getting into what I know everybody has been winning for, and that is deep learning.

So, up to this point, we've got just a simple rule based a I, even though it's just if statements some people who seemed confused, it is still a I Ah, but what we're gonna be doing now is implementing deep learning now, right out of the gate.

I just want to say, First of all, this is a completely unsolved problem.

No one has solved.

You know, Starcraft tuned by solved.

I mean, you know, defeated expert players.

Let's say no one has done that.

They have done it in s'more simplified games like, you know, if we can remove a lot of complexity from the game, we can definitely defeat human players already, but so far that the actual game itself in its entirety, um, you know, allowing let's say, us tow, tow, build all kinds of different units, build different buildings, research, different upgrades and stuff like that that has not been defeated because it's just so complicated.

So, uh so, first of all, let's get that out of the way.

Second of all, I do not know the answer to this problem.

All this is going to be at this point.

Forward is my attempt at solving Starcraft, who is deep learning.

All right, so how are we going to do it?

So the first thing is, especially with an unsolved task like this is there is there is no tutorial that says, Oh, use this network or whatever.

We have to figure out how we're going to solve it.

So I ran out of the gate.

When I have a problem like this, the first thing we got to think about is what you know, what do we hope to achieve?

But then how can we do it?

So we know we want to use deep learning.

Um, but first we need to just, like, dramatically simple simplify what's going on here.

And at least for me, I think the weakest point in our bots at the moment and, um, a relatively easy task would be the attack method here, so I can simplify the attack to be you know, Don't attack, Defend our base, attacked the enemy base or attack any buildings.

Attack enemy units so I can break that down into four really simple choices.

All right.

And then we could take those choices, and we can either reply them unit, um, you know, by unit or by entire army or entire Idol army.

So entire army.

That's not already performing some objective.

So I think the best starting point it would be to apply it to the entire idol army.

Then, um maybe later apply it.

You know, on a per unit basis, I think that would result in really cool things, but I think it will overly complicate things to start.

Okay, so we got that another way.

We know what we want the output of our neural network to be.

Now, what's the input?

What's the structure of our neural network?

Well, I know a lot of people are going to be like I want reinforcement learning and the only reinforcement learning everyone seems to know about its cue learning.

I'm not gonna do que learning here.

I think you learning is way too going to be way too complicated for this kind of a challenge where you take a path.

And then if you got the reward, you, you know, divvy up that reward amongst all the steps you took along the way in your path.

First of all, it's gonna result in a huge amount of training data like but but not really meaningful.

It's just gonna be super noisy.

But also, it's just it's not really the best solution.

I don't think here.

So other than being click baby, I don't think you learning is actually how we're gonna solve this again.

I don't have the answers.

Just don't think it's gonna be that, um there are other forms of reinforcement learning surprise.

But I think what I want to go with is more like an evolutionary learning method.

I think that's most fitting here, too.

So the idea is to have an aye aye, that makes choices.

And if it makes good choices and it wins the war, well, congratulations.

You get to move on your become a continued asset to the gene pool, and if you lose the battle well, you're forgotten history.

You don't get to reproduce into the future.

So I think that makes a little more sense here, and especially if we are able to get down to it.

You know, unit by unit or something like that could be kind of cool, but really, the victorious algorithm either wins or loses as a whole.

So anyways, that's the way I want to do it.

Okay, so that's kind of the structure of how I want to do things we still have to talk about.

Okay.

What is the neural networks that we're gonna use?

Because basically, what kind of input do we want?

Because basically, when you choose a neural network, you gotta think about what kind of input we're gonna put in.

So again, we're just don't We're going from the end forward here.

Um, starting with you know what did we want our neural network to output?

What are we going, Teoh?

You know, what are our hopes for in terms of how it's gonna actually learn.

And, um yeah, basically, learn.

Now, what are we gonna put into this neural network?

And that's going thio basically force our hand was what kind of girl that we're going to use.

And the issue here with Starcraft two is there's so many variables that are.

You know, there's almost a variable number that well, there is a variable number of variables, and even our variables have variables.

So you know, each unit is a variable, but then the type of unit And where is that unit?

All those things come into play.

That's just very complex.

It's very hard to feed it into a neural network so immediately.

We're looking at some pretty complex networks.

So to me, I think the simplest network that you can pass any amount of data and any variable data to would be a convolution neural network because you could visualize everything.

So, first of all, you can visualize the unit.

You have a color code units based on what they are, you concise eyes, them, uh, you can also visualize things like our our materials.

Like, how much minerals do we have in the gas that we have you?

Could you visualize that with a bar or something like that?

And the confident should be able to figure that out.

So that's my plan going in, and the first thing that we have to do is basically well, there's two things we have to kind of modify this a little bit.

We wantto further simplify because you just got to make things as simple as possible to start.

If you're successful, make it a little more complex.

So the first thing I want to do is I just I don't want the stalker's anymore.

We're gonna go with void raise later on.

We're pride gonna need stockers.

Because if we only do void raise, we really can't defend ourselves against early rush is, um so just keep that in mind.

Otherwise, I think I'm gonna leave the attack method the same because we're not going to build stalkers anymore.

We don't.

We only need one gateway.

And we only need that gateway to build the cybernetics core, which we need for the Star Gate.

So, um, so rather than this logic, I'm just gonna get rid of this and make that one.

So if we have less than one gateway well, let's build one.

Otherwise, who cares?

And then we don't want to do 1/2.

We'd like to just build Well, let's just get rid of the divided by two, and then these extra parentheses here.

All right, so, um Oh, let's see.

And then Bill, defensive force.

We only want to build the void raise.

Okay, so that's the simplified logic.

So now for our attack, we only have one type of unit.

Um, and all that again, we can make that more complex.

I don't see any reason when we use the confident that we can't color, code and size.

The stalker's a little different.

Um, I don't see any problem with that.

It's just right now, I want to keep things as simple as possible.

If I find this to be successful, we can try toe, add other variables and see if that you know, is acceptable or note that throws the whole thing off and it can't figure it out or whatever.

So the first thing we always want to do in a case like this, it's just let's make this is easy as possible to learn for a neural network.

So now that we've got that, the next thing that we need to do is one modify the attack method to attack based on some sort of rules to start, they're just gonna be random, and then we're gonna let random, um, you know, with just run a ton of games and anytime random wins.

Well, we're gonna take the set of actions that random took, and we're going to say, Hey, these were these were good actions.

Good job, army.

And we're gonna use that.

His training data, Any of the armies that lose randomly, um, we're gonna use that Won't be training data now if what we're gonna do is against the easy baht, Um, you could just win randomly for no good reason against the hard.

But I'm pretty confident if you do things smart or someone intelligently, you'll win.

And if you don't, you'll lose.

So my guess is this will result in good training data again.

I don't know.

After ideo, So come along.

Okay, so So we're gonna need to modify that.

But first, let's do the do the input.

So what is the imagery that we're going to see?

So to begin, we're gonna use open TV just for a visual ization sake.

Um, you really don't actually?

Well, yeah, you should have a baby.

You're gonna need it because we're going to use open CVS drawing methods and stuff like that.

So we're gonna import a CVI to an important numb pie.

You don't have these numb pie should be relatively simple to install.

It's very much like all your other packages.

CV to isn't not if you're on windows.

If you are on Windows, head on over to place and person in it and do a search for open sea V.

Okay.

And hopefully you'll find in that first tutorial how to install it on Windows and all is well, is like clinics and stuff like that's a little easier on Lenox windows.

You just need to go to that unofficial sources.

Okay, So once we have that, we're gonna create a new method, and I'm gonna call this new method Intel.

So the intel method, uh, others define or making right here a sink.

Let me zoom in a bit.

Define Intel itself so and tell will just be the visual ization of what's going on in the game.

So right now we're gonna I am gonna hand right out at least the basic drawing of a thing.

Uh, and then when we start to build on this, I'm gonna do the copy pasta because it will be anything new.

So, um, so the first thing is ah, to draw.

We're just gonna draw basically the game map.

We have access to game information.

And so if you did, like self dot game info, uh, and then game info has dot map size.

Now, how do I figure out all these things?

Right.

Um, there is a wiki for pious see, too.

But it's, you know, not really the best week you have ever seen.

And that's okay.

No insult.

I really like pie asi to thank you for making it.

Because I'm sorry, Python Dash s c two because pious he, too was just impossible to understand.

So anyways, gave.

That's what I was saying.

How do I figure this stuff out?

Honestly, um, you can immediately like you could go to the source code.

The other thing you can do because a lot of times in the source code, things inherit from other things and global bond, it gets really hard to follow.

You could always just print dir self and just hope they make useful names and you can see, like, what are all the methods and just kind of read through him, um, or you go to the source code and compile the list by reading it or whatever.

I just know that that's honestly what I do.

I just go through there in and see what my options are.

And that's how I found game size.

I just figured it would be in there.

So, um, so we're gonna say is game data and this will be the initial image.

And we're just gonna make that NPD zeroes and then an MPD zeroes.

You pass a shape.

So in the shape of our NPR zeros, um, it's going to be so the shape will be to pull basically, that will be self dot game info dot map, suck, map size and then one first ith and then the zero with Oh my gosh, what is hairs and up to?

What is he up to?

So Internet array, its height by width, but in the image it's with by height.

So we're going to reverse thes because we're using an array for an image and we kind of wanna flip.

Okay.

Next thing is it is not just X and why they also have color that took an embarrassing law embarrassingly long time for me to figure out what the heck was going wrong on.

Then this will be our data type.

So, uh Okay, so that's game data.

Now, let's say we want to draw stuff.

So let's say four nexus in self units of nexus type.

We want to draw them.

So we're going to say the next is let's say next position is nexus stop posts.