Name: 強化学習。クラッシュコースAI#9 (Reinforcement Learning: Crash Course AI#9)
Uploaded: 2021-01-14T10:29:03.000Z
Duration: 11 min 28 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

Hey, I'm Jabril and welcome to Crash Course AI.

Say I want to get a cookie from a jar that's on a tall shelf.

There isn't one “right way” to get the cookies.

Maybe I find a ladder, use a lasso, or build a complicated system of pulleys.

These could all be brilliant or terrible ideas, but if something works, I get the sweet taste

of victory... and I learn that doing that same thing could get me another cookie in

We learn lots of things by trial-and-error, and this kind of “learning by doing” to

achieve complicated goals is called Reinforcement Learning.

So far, we've talked about two types of learning in Crash Course AI: Supervised Learning,

where a teacher gives an AI answers to learn from, and Unsupervised Learning, where an

Reinforcement Learning is particularly useful for situations where we want to train AIs

to have certain skills we don't fully understand ourselves.

For example, I'm pretty good at walking, but trying to explain the process of walking

What angle should your femur be relative to your foot?

And should you move it with an average angular velocity of… yeah, never mind… its really

With reinforcement learning, we can train AIs to perform complicated tasks.

But unlike other techniques, we only have to tell them at the very end of the task if

they succeeded, and then ask them to tell us how they did it.

(We're going to focus on this general case, but sometimes this feedback could come earlier.

So if we want an AI to learn to walk, we give them a reward if they're both standing up

and moving forward, and then figure out what steps they took to get to that point.

The longer the AI stands up and moves forward, the longer it's walking, and the more reward

So you can kind of see how the key to reinforcement learning is just trial-and-error, again and

For humans, a reward might be a cookie or the joy of winning a board game.

But for an AI system, a reward is just a small positive signal that basically tells it “good

Google Deepmind got some pretty impressive results when they used reinforcement learning

to teach virtual AI systems to walk, jump, and even duck under obstacles.

It looks kinda silly, but works pretty well!

Other researchers have even helped real life robots learn to walk.

So seeing the end result is pretty fun and can help us understand the goals of reinforcement

But to really understand how reinforcement learning works, we have to learn new language

to talk about these AI and what they're doing.

Similar to previous episodes, we have an AI (or Agent) as our loyal subject that's going

An agent makes predictions or performs Actions, like moving a tiny bit forward, or picking

And it performs actions based on its current inputs, which we call the State.

In supervised learning, after /each/ action, we would have a training label that tells

our AI whether it did the right thing or not.

We can't do that here with reinforcement learning, because we don't know what the

“right thing” actually is until it's completely done with the task.

This difference actually highlights one of the hardest parts of reinforcement learning

It's hard to know which actions helped us get to the reward (and should get credit)

and which actions slowed down our AI when we don't pause to think after every action.

So the agent ends up interacting with its Environment for a while, whether that's

a game board, a virtual maze, or real life kitchen.

And the agent takes many actions until it gets a Reward, which we give out when it wins

a game or gets that cookie jar from that really tall shelf.

Then, every time the agent wins (or succeeds at its task), we can look back on the actions

it took and slowly figure out which game states were helpful and which weren't.

During this reflection, we're assigning Value to those different game states and deciding

We need Values and Policies to get anything done in reinforcement learning.

Let's say I see some food in the kitchen: a box, a small bag, and a plate with a donut.

So my brain can assign each of these a value, a numerical yummy-ness value.

The box probably has 6 donuts in it, the bag probably has 2, and the plate just has 1…

Now that I've assigned each of them a value, I can decide on a policy to plan what action

The simplest policy is to go to the highest value (that box of possibly 6 donuts).

But I can't see inside of it, and that could be a box of bagels, so it's high reward

Another policy could be low reward but low risk, going with the plate with 1 guaranteed

Personally, I'd pick a middle-ground policy, and go for the bag because I have a better

chance of guessing that there are donuts inside than the box, and a value of 1 donut isn't

That's a lot of vocab, so let's see these concepts in action to help us remember everything.

Our example is going to focus on a mathematical framework that could be used with different

Let's say John-Green-bot wants to go to the charging station to recharge his batteries.

In this example, John-Green-bot is a brand new Agent, and the room is the Environment

From where he is now in the room, he has four possible Actions: moving up, down, left, or

And his State is a couple of different inputs: where he is, where he came from, and what

For this example, we'll assume John-Green-bot can see the whole room.

So when he moves up (or any direction), his state changes.

But he doesn't know yet if moving up was a good idea, because he hasn't reached a

He found the battery, so he got a Reward (that little plus one).

Now, we can look back at the path he took and give all the cells he walked through a

Value -- specifically, a higher value for those near the goal, and lower for those farther

These higher and lower values help with the trial-and-error of reinforcement learning,

and they give our agent more information about better actions to take when he tries again!

So if we put John-Green-bot back at the start, he'll want to decide on a Policy that maximizes

字幕リスト動画再生

強化学習。クラッシュコースAI#9 (Reinforcement Learning: Crash Course AI#9)

strategy

guarantee

completely

impressive