Name: Strike a Pose - MLとArduinoを使ったJavaScriptのジェスチャー認識 - Charlie Gerard - JSConf US 2019 (Strike a Pose - Gesture recognition in JavaScript w/ ML & Arduino - Charlie Gerard - JSConf US 2019)
Uploaded: 2021-01-14T10:22:31.000Z
Duration: 27 min 53 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

命令形

Strike a pose  gesture recognition in JavaScript with Machine Learning & Arduino

Charlie Gerard PARISS: I have been enjoying all of these

動名詞

presentations from backstage and I'm like reading everything backwards.

So, next we have strike a pose  gesture recognition in JavaScript with Charlie Gerard.

So, can we give some encouragement and give some clapping and  whoo!

And thanks, everybody, for the warm welcome.

We'll see if you feel the same way at the end of the talk.

You might not know Atlassian, but maybe the product, Jira.

But what I'm going to talk about today has nothing to do with Jira.

And outside of my job as well, I'm a Google developer expert in web technologies and a

There's been a few other speakers part of these groups in the past few days.

If you have questions, feel free to ask us.

But the title of this talk was strike a pose.

And I don't know if there's any Madonna fans.

So, I really like human computer interaction.

But at home, I like to explore ways to interact with technology and interface with devices

Maybe not with your computer or your phone, or maybe using the phone a different way.

So, this is why I called this talk strike a pose because it's from the song Vogue.

And I don't know if you know it, but vogueing is a type of dance where you don't follow

You just express yourself in different gestures and it's who you are and mow with dance.

But we are going use our gestures to play a game of Street Fighter.

So, to do this, there's a few different ways.

But the way we're going to talk about today, you need this to get started.

I'm really glad some of you did workshops yesterday to use hardware.

You already know a little bit what I'm going to talk about.

This is a maker 1000 that connects via Wi Fi.

On top that have, we're going to be holding that piece of hardware and using machine learning

to find patterns in the data that we're tracking in our movements and we're going to do absolutely

And for TensorFlow, we're going to use TensorFlow JS.

We want to be able to play Street Fighter with our body movements.

What I mean by that is when I want to punch I'm going to actual punch.

I have punch, and  let's just call them like that.

So, I used a Wi Fi board because if you use a basic board like the UNO, it has to be timed

to the computer and I want to be able to move around.

If I wanted to, I could play from the back of the room as well.

But the Arduino is going to render the program, but there's no built in sensor to track motion.

We are going to add an NPU1650, an accelerometer and gyro scope.

I'm going to become an NPU1650 right now.

If you can track motions in three different axis, X, Y and Z.

And accelerometer, left to right, that would be the X axis and then the Y axis and then

But if you use the accelerometer just by itself, you're not walking like this through life.

We use the gyro scope to get the changes of rotation.

We have to track three more axis, but with rotation.

And in the end, we're going to have six points of data to track over time as we are moving.

But there is one last thing that I did as well, which is, oh, sorry, a button.

And I did a button because I want to track the data only when I am doing a gesture.

You don't have to do that, but as a first prototype, it was easier to only get the data

from my gesture to train the algorithm to only  to only find patterns in the data when

When I'm gonna hold the button, do the gesture.

And when I release it, take into consideration that data coming from the sensor.

But in terms of code we start by always requiring a few normal.

Johnny Five, the system, and the client to set up our board.

As it is a Wi Fi board, you have to connect to the IP address of the Arduino and a certain

And then once the board is ready, once that is set up, we set the pin that you want, in

this case, A0, and then we create a file, like a stream, to be able to write data to

In this code sample, I had coded a name of the file, example punch zero.

Because you have to record data a lot of times.

And this is how I spend my personal time.

After you open the stream to write to a file, you have to instantiate with a sensor the

Strike a Pose - MLとArduinoを使ったJavaScriptのジェスチャー認識 - Charlie Gerard - JSConf US 2019 (Strike a Pose - Gesture recognition in JavaScript w/ ML & Arduino - Charlie Gerard - JSConf US 2019)

字幕リスト動画再生

Strike a Pose - MLとArduinoを使ったJavaScriptのジェスチャー認識 - Charlie Gerard - JSConf US 2019 (Strike a Pose - Gesture recognition in JavaScript w/ ML & Arduino - Charlie Gerard - JSConf US 2019)

stuff

accurate

applause

expect