Placeholder Image

字幕表 動画を再生する

  • [DING]

  • Hello.

  • And welcome to a new tutorial series

  • on The Coding Train about a piece of software

  • called Runway.

  • So what is Runway?

  • How do you download and install Runway and kind of tinker

  • around with it?

  • That's all I'm going to do in this particular video.

  • Now, let m be clear, Runway is not something that I've made.

  • Runway is made by a company, a new company

  • called Runway itself.

  • And it's a piece of software.

  • You can use it and download it for free.

  • You can use it for free.

  • There are aspects of it that require Cloud GPU credits,

  • which I'll get into later.

  • And you can get some free credits and a coupon code

  • that you'll find in the description of this video.

  • But really I want to just talk to you

  • about what it is cause I'm so excited about it,

  • and I'm planning to use it in the future,

  • in a lot of future tutorials and coding challenges, and teaching

  • things that I'm going to do.

  • And I also should just mention that I

  • am an advisor to the company Runway itself.

  • So I'm involved in that capacity.

  • All right.

  • So what is Runway?

  • Right here it says machine learning for creatives.

  • Bring the power of artificial intelligence

  • to your creative projects with an intuitive and simple

  • visual interface.

  • Start exploring new ways of creating today.

  • So this, to me, is the core of Runway.

  • I am somebody who's a creative coder.

  • I'm working with processing and P5JS.

  • You might be working with other pieces of software.

  • That's just commercial software, coding environments.

  • You're writing your own software.

  • And you want to make use of recent advances

  • in machine learning.

  • You've read about this model.

  • You saw this YouTube video about this model.

  • Can you use it in your thing?

  • Well, before Runway one of the things you might have done

  • is find your way to some GitHub repo that

  • had like this very long ReadMe about all

  • the different dependencies you need to install and configure.

  • And then you've got to download this and install this, and then

  • build this library.

  • And you can really get stuck there for a long time.

  • So Runway is an all in one piece of software

  • with an interface that basically will run machine learning

  • models for you, install and configure them

  • without you having to do any other work

  • but press a button called Install.

  • And it gives you an interface to play with those models,

  • experiment with those models, and then broadcast

  • the results of those models to some other piece of software.

  • And there's a variety of ways you

  • can do that broadcasting, through HTTP requests,

  • through OSC messages.

  • And all these things might not make sense

  • to you, which is totally fine.

  • I am going to poke through them and show you

  • how they work, with an eye towards at least showing you

  • how to pair Runway with processing,

  • and how to pair Runway with P5JS,

  • and I'll also show you where there's lots of other examples

  • and things you can do with other platforms, and stuff like that.

  • So the first step you should do is click here

  • under Download Runway Beta.

  • It will automatically trigger a download

  • for Mac OS, Windows, or Linux.

  • I've actually already downloaded and installed Runway.

  • So I'm going to kind of skip that step,

  • and just actually now run the software.

  • Ah.

  • And now it's saying, welcome to Runway.

  • Sign in to get started.

  • OK.

  • So if you already have an account,

  • you could just sign in with your account.

  • I do already have an account.

  • But I'm going to create a new one, just so we can

  • follow along with the process.

  • So I'm going to go here.

  • Create an account.

  • I'm going to enter my email address, which is-- shh.

  • Don't tell anyone-- daniel@thecodingtrain.com.

  • Then I'm going to make a username and password.

  • Now that I've put in my very strong password,

  • I'm going to click Next.

  • And I'm going to give my details, Daniel Schiffman,

  • The Coding Train.

  • Create account.

  • Ah.

  • And it's giving me a verification code

  • to daniel@thecodingtrain.com.

  • Account has now been created, and I can click Start.

  • So once you've downloaded, installed Runway, and signed up

  • for an account, logged into your account,

  • you will find this screen.

  • So if you've been using Runway for a while,

  • you might then end up here, clicking on open workspaces,

  • because workspaces are a way of collecting

  • a bunch of different models that you

  • want to use for a particular project into a workspace.

  • But we haven't done any of that.

  • So the first thing that I'm going to do

  • is just click on Browse Models.

  • So the first thing that I might suggest that you do

  • is just click on a model and see what

  • you can do to play with it in the Runway interface itself,

  • because one of the things that's really wonderful about Runway

  • is as a piece of software and an interface you can explore

  • and experiment with the model to understand how it works,

  • what it does well, what it doesn't do well,

  • what it does at all, before starting

  • to bring it into your own software or your own project.

  • So I'm going to pick this Spade Coco model, which I have never

  • looked at before.

  • This is very legitimate me.

  • I have no idea what's going to happen when I click on that.

  • And now, here I can find out some more information

  • about the model.

  • So I could find out what does the model do?

  • It generates realistic images from sketches and doodles.

  • I can find out more information about the model.

  • For example, this is the paper that describes this model,

  • "Semantic Image Synthesis with Spatially Adaptive

  • Normalizations Trained on COCO-Stuff Data Set."

  • Remember when someone asked, is this a tutorial for beginners.

  • Well, it is for beginners in that you're a beginner.

  • You can come here and play around with it.

  • But you can go very deep too if you want to find the paper,

  • read through the notes, and understand

  • more about this model, how it was built,

  • what data it was trained on, which is always

  • a very important question to ask whenever you're

  • using a machine learning model.

  • So we can see there are attributions here.

  • So this is the organization that trained the model.

  • These are the authors of the paper.

  • We can see the size of it, when it was created,

  • if it's CPU and GPU supported.

  • We could also go under Gallery.

  • And we can see just some images that have been created.

  • So we can get an idea.

  • This is a model that's themed around something

  • called image segmentation.

  • So I have an image over here.

  • What does it mean to do image segmentation?

  • Well, this image is segmented, divided into a bunch

  • of different segments.

  • Those segments are noted by color.

  • So there's a purple segment, a pink segment,

  • a light green segment.

  • And those colors are tied to labels in the model,

  • essentially, that know about a kind of thing

  • that it could draw in that area.

  • So you could do image segmentation in two ways.

  • I could take an existing image, like an image of me,

  • and try to say, oh, I'm going to segment it.

  • This is where my head is.

  • This is where my hand is.

  • This is where my hand is.

  • Or I could generate images by sort

  • of drawing on a blank image, saying put a hand over here.

  • Put a head over here.

  • So that's what image segmentation

  • is, at least in the way that I understand it.

  • What have I done so far?

  • I've downloaded Runway.

  • I've poked around the models.

  • And I've just clicked on one.

  • Now, I want to use that model.

  • I want to play with it.

  • I want to see it run.

  • So I'm going to go here to Add to Workspace.

  • It's right up here.

  • Add to Workspace.

  • Now, I don't have a workspace yet.

  • So I need to make one.

  • And I'm going to call this workspace,

  • I'm going to say Coding Train Live Stream.

  • So I'm going to do that.

  • I'm going to hit Create.

  • Now, I have a workspace.

  • You can see, this is my workspace.

  • I have only one model added to this workspace over here.

  • And it's kind of highlighting up for me right now what to do.

  • I need to choose an input source.

  • So every machine learning model is different.

  • Some of them expect text input.

  • Some of them expect image input.

  • Some of them might expect input that's

  • arbitrary scientific data from a spreadsheet.

  • Then the model is going to take that input in, run it

  • through the model, and produce an output.

  • And that output might be numbers.

  • Or it also might be an image.

  • Or it might be more text.

  • So now we're in sort of the space of a case by case basis.

  • But if I understand image segmentation correctly,

  • I'm pretty sure the input and the output

  • are both going to be an image.

  • Let's make a little diagram.

  • So we have this--

  • what was this model called again?

  • Spade Coco.

  • So we have this machine learning model.

  • Presumably there's some neural network architecture in here.

  • Maybe it has some convolutional layers.

  • This is something we would want to read that paper

  • to find out more.

  • Runway is going to allow us to just use it out of the box.

  • And I certainly would always recommend

  • reading more about this to learn more about how to use it.

  • So my assumption here is in my software that I want to build,

  • I want to maybe create a drawing piece of software

  • that allows a user to segment down an image.

  • So you can imagine maybe I'm going to kind of draw

  • something that's one color.

  • Look, I could use different colored markers.

  • I'm going to sort of fill this image in with a bunch

  • of different colors.

  • And then I am going to feed that into the model.

  • And out will come an image.

  • So we have input.

  • And we have output.

  • And again, this is going to be different for every model

  • that we might pick in Runway.

  • Although, there's a lot of conventions.

  • A lot of the models expect images

  • as input and output images.

  • Some of them expect text as input, and output an image,

  • or image as input and output text.

  • Et cetera.

  • And so on and so forth.

  • And so now what I want to do is choose the input source

  • in Runway for the model.

  • So something that's going to produce a segmented image.

  • So that could be coming from a file.

  • It could actually come from a network connection, which

  • I'll get into maybe in a future video,

  • or you can explore on your own.

  • I'm just going to pick segmentation.

  • I know.

  • This is like the greatest thing ever.

  • Because what's just happened is image segmentation

  • is a common enough feature of machine learning models

  • that Runway has built into it an entire drawing engine so

  • that you can play around with image segmentation.

  • And you can see, these are the colors for different labels.

  • So it looks like it's a lot of transportation stuff.

  • So maybe what I want is let's try

  • let's try drawing some people.

  • [MUSIC PLAYING]

  • Two people with an airplane and a wineglass flying overhead.

  • OK.

  • How are we doing?

  • Now, I'm going to choose an output.

  • And I just want to preview.

  • Right?

  • Cause preview right now is, I don't need to export this.

  • I don't need to use it somewhere else.

  • I just want to play around with it in Runway itself.

  • So I'm going to hit Preview.

  • Now I have selected my input, which is just the segmentation

  • interface of Runway itself.

  • I have selected my output, which is just a preview.

  • Now, it's time for me to run the model.

  • And here we go.

  • Run Remotely.

  • So remote GPU enabled.

  • And you can see, just by signing up

  • for Runway I have $10 in remote GPU credits

  • It'll be interesting to see how much just running this once

  • actually uses.

  • So one thing I'll mention now, if you

  • want to get additional credits, I can go over here.

  • This is like the sort of icon for my profile.

  • I can click on this.

  • I'm going to go now to here.

  • I'm going to go to Get More Credits.

  • And this is going to take me to a browser page.

  • And I could certainly pay for more credits.

  • But I'm going to click here.

  • And I'm going to redeem credits by saying CODINGTRAIN

  • right here.

  • So if you would like to get an additional $10 in credits,

  • you can do this.

  • And we can see now I should have $20 in credits.

  • So this icon up here, just so we're clear,

  • this icon up here is your workspaces,

  • of which I only have one with one model that's

  • connected to a remote GPU.

  • And if I wanted to look at other models,

  • I would go here to this icon.

  • All right.

  • Now, I'm going to press Run Remotely.

  • [DRUM ROLL]

  • Running the model remotely.

  • Whoa!

  • [TA-DA]

  • Oh, my.

  • Oh, it is so beautiful.

  • Mwuah.

  • I cannot believe it.

  • So this is what the Spade Coco machine learning

  • model generate.

  • It's really interesting to see the result here.

  • So you could think, me knowing nothing

  • about this model, kind of how it works and what to expect,

  • you get some pretty weird results with it.

  • Probably if I were a bit more thoughtful, maybe if I even

  • filled in the entire space--

  • I probably left so much of it blank,

  • and also included a giant wine glass with two people.

  • It's kind of creepy looking.

  • Although, I think this sort of resembles me

  • in some strange sort of way.

  • And we can see here.

  • Look at this.

  • $0.05.

  • So one thing I should mention is the reason

  • why that took a long time, it was spinning up

  • the server and everything to start actually

  • running the model.

  • But now that it's running in real time,

  • it can happen much more quickly.

  • So let's try filling it.

  • So what would be a good thing to fill it with?

  • Let's try floor wood.

  • Let's try filling it with wood floor.

  • Oh, whoa.

  • Then let's try to put some fruit.

  • Ooh.

  • This is looking much better now.

  • Let's put an orange next to it.

  • Let's put a couple oranges and make a little bowl of fruit.

  • Wow.

  • This is crazy.

  • Wow.

  • I got to stop.

  • That's pretty amazing.

  • So again, here was just a little moment later

  • of being a little more thoughtful to think about how

  • this model actually works.

  • And if I looked at the data set, which is fairly well-known,

  • I imagine, Coco image data set, then

  • that's probably going to give me even more information

  • to think about what it's going to do well.

  • But you can see how it's able to sort of see

  • a little pile of fruit here on a wood background.

  • It almost looks a little more like cloth,

  • like it's sitting on a table.

  • Pretty realistic.

  • And yes.

  • Charlie England points out, which is correct,

  • this is continuing to use the GPU credits.

  • And we can see that still, though,

  • even with doing a bunch of live painting,

  • I've just used $0.10 there.

  • So you can do a lot with the free $10,

  • just in playing around.

  • So input wise, I chose to do segmentation here.

  • But I could also use a file.

  • So if I wanted to open a file on the computer,

  • I could do it that way.

  • And then output, if I change to export,

  • I could also actually export that

  • to a variety of different formats.

  • But, of course, I could also right here just

  • under Preview I can click this Download Save button.

  • And now I am saving forever more this particular image

  • as a file.

  • Now, what's really important here actually,

  • more important here, is under Network.

  • So if what I wanted to do was click over here under Network,

  • this means I can now communicate with this particular machine

  • learning model from my own software.

  • Whether that's software that I've downloaded or purchased

  • that somebody else has made that speaks

  • one of these particular protocols,

  • or my own software that I'm writing in

  • just about any programming language or environment

  • if you have a framework, or module, or a library,

  • or support these types of protocols.

  • And one of the nice things here, if I click on JavaScript,

  • we can see there's actually a bit of code here that you can

  • actually just copy/ paste into your JavaScript to run it

  • directly.

  • So I'm going to come back.

  • OSC is also a really popular messaging network protocol

  • for creative coders.

  • It stands for Open Sound Control and allows

  • you to send data between applications.

  • So I will also kind of come back in a separate video

  • and show you about how some of these work.

  • I should also probably mention that your Runway software

  • itself works in a very similar way to a piece of software

  • called Wekinator that you might be familiar with.

  • Wekinator is a software that was created by Rebecca Fiebrink

  • years ago that allows you to train

  • a neural network with data sent over OSC messaging,

  • and then get the results of that after the fact.

  • Though, I think the real sort of key difference here is Runway

  • is really set up to support a huge treasure

  • trove of pre-trained models.

  • Whereas Wekinator was more for training neural networks

  • on the fly with small bits of data.

  • And I will say that one of the things that Runway is planning,

  • maybe by September, is to start coming out

  • with features for training your own model as well.

  • So thanks for watching this introduction to Runway,

  • just sort of the basics of downloading and installing

  • the software, what it is from a high level point of view, what

  • features of the interface work, how to get some free cloud

  • credits.

  • And what I would suggest that you

  • do after watching this video is download, run the software,

  • and go to this Browse Models page.

  • So you can see, there's a lot of different models

  • for looking at motion, generative, community, text,

  • recognition.

  • Click around here.

  • Let's try this recognition one.

  • Face recognition.

  • Dense cap.

  • Where is PoseNet in here?

  • That might be under motion?

  • DensePose PoseNet.

  • So here's a model called PoseNet which

  • performs real time skeletal tracking of one or more people.

  • I've covered this model in other libraries,

  • like the ML5 JS library with TensorFlow JS.

  • And so what I'm going to do in the next video

  • is use this model, PoseNet, in Runway with my webcam,

  • running it locally on this computer

  • without requiring cloud credits, and then

  • get the results of this model in [? processing ?] itself.

  • So I'm going to show you that whole workflow.

  • But poke around.

  • Click around.

  • Find a model that you like.

  • Let me know about it in the comments.

  • Share images that you made with it.

  • And I look forward to seeing what you make with runway.

  • Great.

  • Thanks for watching.

  • [MUSIC PLAYING]

[DING]

字幕と単語

ワンタップで英和辞典検索 単語をクリックすると、意味が表示されます

B1 中級

ランウェイ入門。クリエイターのための機械学習(前編 (Introduction to Runway: Machine Learning for Creators (Part 1))

  • 0 0
    林宜悉 に公開 2021 年 01 月 14 日
動画の中の単語