字幕表 動画を再生する
-
[DING]
-
Hello.
-
And welcome to a new tutorial series
-
on The Coding Train about a piece of software
-
called Runway.
-
So what is Runway?
-
How do you download and install Runway and kind of tinker
-
around with it?
-
That's all I'm going to do in this particular video.
-
Now, let m be clear, Runway is not something that I've made.
-
Runway is made by a company, a new company
-
called Runway itself.
-
And it's a piece of software.
-
You can use it and download it for free.
-
You can use it for free.
-
There are aspects of it that require Cloud GPU credits,
-
which I'll get into later.
-
And you can get some free credits and a coupon code
-
that you'll find in the description of this video.
-
But really I want to just talk to you
-
about what it is cause I'm so excited about it,
-
and I'm planning to use it in the future,
-
in a lot of future tutorials and coding challenges, and teaching
-
things that I'm going to do.
-
And I also should just mention that I
-
am an advisor to the company Runway itself.
-
So I'm involved in that capacity.
-
All right.
-
So what is Runway?
-
Right here it says machine learning for creatives.
-
Bring the power of artificial intelligence
-
to your creative projects with an intuitive and simple
-
visual interface.
-
Start exploring new ways of creating today.
-
So this, to me, is the core of Runway.
-
I am somebody who's a creative coder.
-
I'm working with processing and P5JS.
-
You might be working with other pieces of software.
-
That's just commercial software, coding environments.
-
You're writing your own software.
-
And you want to make use of recent advances
-
in machine learning.
-
You've read about this model.
-
You saw this YouTube video about this model.
-
Can you use it in your thing?
-
Well, before Runway one of the things you might have done
-
is find your way to some GitHub repo that
-
had like this very long ReadMe about all
-
the different dependencies you need to install and configure.
-
And then you've got to download this and install this, and then
-
build this library.
-
And you can really get stuck there for a long time.
-
So Runway is an all in one piece of software
-
with an interface that basically will run machine learning
-
models for you, install and configure them
-
without you having to do any other work
-
but press a button called Install.
-
And it gives you an interface to play with those models,
-
experiment with those models, and then broadcast
-
the results of those models to some other piece of software.
-
And there's a variety of ways you
-
can do that broadcasting, through HTTP requests,
-
through OSC messages.
-
And all these things might not make sense
-
to you, which is totally fine.
-
I am going to poke through them and show you
-
how they work, with an eye towards at least showing you
-
how to pair Runway with processing,
-
and how to pair Runway with P5JS,
-
and I'll also show you where there's lots of other examples
-
and things you can do with other platforms, and stuff like that.
-
So the first step you should do is click here
-
under Download Runway Beta.
-
It will automatically trigger a download
-
for Mac OS, Windows, or Linux.
-
I've actually already downloaded and installed Runway.
-
So I'm going to kind of skip that step,
-
and just actually now run the software.
-
Ah.
-
And now it's saying, welcome to Runway.
-
Sign in to get started.
-
OK.
-
So if you already have an account,
-
you could just sign in with your account.
-
I do already have an account.
-
But I'm going to create a new one, just so we can
-
follow along with the process.
-
So I'm going to go here.
-
Create an account.
-
I'm going to enter my email address, which is-- shh.
-
Don't tell anyone-- daniel@thecodingtrain.com.
-
Then I'm going to make a username and password.
-
Now that I've put in my very strong password,
-
I'm going to click Next.
-
And I'm going to give my details, Daniel Schiffman,
-
The Coding Train.
-
Create account.
-
Ah.
-
And it's giving me a verification code
-
to daniel@thecodingtrain.com.
-
Account has now been created, and I can click Start.
-
So once you've downloaded, installed Runway, and signed up
-
for an account, logged into your account,
-
you will find this screen.
-
So if you've been using Runway for a while,
-
you might then end up here, clicking on open workspaces,
-
because workspaces are a way of collecting
-
a bunch of different models that you
-
want to use for a particular project into a workspace.
-
But we haven't done any of that.
-
So the first thing that I'm going to do
-
is just click on Browse Models.
-
So the first thing that I might suggest that you do
-
is just click on a model and see what
-
you can do to play with it in the Runway interface itself,
-
because one of the things that's really wonderful about Runway
-
is as a piece of software and an interface you can explore
-
and experiment with the model to understand how it works,
-
what it does well, what it doesn't do well,
-
what it does at all, before starting
-
to bring it into your own software or your own project.
-
So I'm going to pick this Spade Coco model, which I have never
-
looked at before.
-
This is very legitimate me.
-
I have no idea what's going to happen when I click on that.
-
And now, here I can find out some more information
-
about the model.
-
So I could find out what does the model do?
-
It generates realistic images from sketches and doodles.
-
I can find out more information about the model.
-
For example, this is the paper that describes this model,
-
"Semantic Image Synthesis with Spatially Adaptive
-
Normalizations Trained on COCO-Stuff Data Set."
-
Remember when someone asked, is this a tutorial for beginners.
-
Well, it is for beginners in that you're a beginner.
-
You can come here and play around with it.
-
But you can go very deep too if you want to find the paper,
-
read through the notes, and understand
-
more about this model, how it was built,
-
what data it was trained on, which is always
-
a very important question to ask whenever you're
-
using a machine learning model.
-
So we can see there are attributions here.
-
So this is the organization that trained the model.
-
These are the authors of the paper.
-
We can see the size of it, when it was created,
-
if it's CPU and GPU supported.
-
We could also go under Gallery.
-
And we can see just some images that have been created.
-
So we can get an idea.
-
This is a model that's themed around something
-
called image segmentation.
-
So I have an image over here.
-
What does it mean to do image segmentation?
-
Well, this image is segmented, divided into a bunch
-
of different segments.
-
Those segments are noted by color.
-
So there's a purple segment, a pink segment,
-
a light green segment.
-
And those colors are tied to labels in the model,
-
essentially, that know about a kind of thing
-
that it could draw in that area.
-
So you could do image segmentation in two ways.
-
I could take an existing image, like an image of me,
-
and try to say, oh, I'm going to segment it.
-
This is where my head is.
-
This is where my hand is.
-
This is where my hand is.
-
Or I could generate images by sort
-
of drawing on a blank image, saying put a hand over here.
-
Put a head over here.
-
So that's what image segmentation
-
is, at least in the way that I understand it.
-
What have I done so far?
-
I've downloaded Runway.
-
I've poked around the models.
-
And I've just clicked on one.
-
Now, I want to use that model.
-
I want to play with it.
-
I want to see it run.
-
So I'm going to go here to Add to Workspace.
-
It's right up here.
-
Add to Workspace.
-
Now, I don't have a workspace yet.
-
So I need to make one.
-
And I'm going to call this workspace,
-
I'm going to say Coding Train Live Stream.
-
So I'm going to do that.
-
I'm going to hit Create.
-
Now, I have a workspace.
-
You can see, this is my workspace.
-
I have only one model added to this workspace over here.
-
And it's kind of highlighting up for me right now what to do.
-
I need to choose an input source.
-
So every machine learning model is different.
-
Some of them expect text input.
-
Some of them expect image input.
-
Some of them might expect input that's
-
arbitrary scientific data from a spreadsheet.
-
Then the model is going to take that input in, run it
-
through the model, and produce an output.
-
And that output might be numbers.
-
Or it also might be an image.
-
Or it might be more text.
-
So now we're in sort of the space of a case by case basis.
-
But if I understand image segmentation correctly,
-
I'm pretty sure the input and the output
-
are both going to be an image.
-
Let's make a little diagram.
-
So we have this--
-
what was this model called again?
-
Spade Coco.
-
So we have this machine learning model.
-
Presumably there's some neural network architecture in here.
-
Maybe it has some convolutional layers.
-
This is something we would want to read that paper
-
to find out more.
-
Runway is going to allow us to just use it out of the box.
-
And I certainly would always recommend
-
reading more about this to learn more about how to use it.
-
So my assumption here is in my software that I want to build,
-
I want to maybe create a drawing piece of software
-
that allows a user to segment down an image.
-
So you can imagine maybe I'm going to kind of draw
-
something that's one color.
-
Look, I could use different colored markers.
-
I'm going to sort of fill this image in with a bunch
-
of different colors.
-
And then I am going to feed that into the model.
-
And out will come an image.
-
So we have input.
-
And we have output.
-
And again, this is going to be different for every model
-
that we might pick in Runway.
-
Although, there's a lot of conventions.
-
A lot of the models expect images
-
as input and output images.
-
Some of them expect text as input, and output an image,
-
or image as input and output text.
-
Et cetera.
-
And so on and so forth.
-
And so now what I want to do is choose the input source
-
in Runway for the model.
-
So something that's going to produce a segmented image.
-
So that could be coming from a file.
-
It could actually come from a network connection, which
-
I'll get into maybe in a future video,
-
or you can explore on your own.