字幕表 動画を再生する
[MUSIC PLAYING]
EWA MATEJSKA: Hi, everybody.
Thank you for joining us.
I'm Ewa Matejska.
I'm a technical program manager with the TensorFlow team.
JIHO CHOI: Hi, I'm Jiho Choi, a software engineer at Google.
And I am very excited to show you the new TensorFlow
profiler.
EWA MATEJSKA: Awesome.
So what is it?
Is it new this year?
JIHO CHOI: Yes.
This is something new we are releasing at this TensorFlow
Dev Summit.
So we will want to run our model faster, right?
And to make it faster, it is essential to collect
the performance data.
And this new TensorFlow profiler enables
you to profile your model and collect such performance data,
and provides you with a set of tools to find and fix
performance problems.
EWA MATEJSKA: What is it supported on?
JIHO CHOI: It supports CUP, GPU, and TPU.
EWA MATEJSKA: Oh, so everything.
Great.
I am excited to see it.
So how do I use it?
JIHO CHOI: So I prepared a quick demo
to show how to use the profiler.
EWA MATEJSKA: So I see a lot of code here.
What is this?
What model are you using?
What data set?
JIHO CHOI: Yeah.
So for this demo, we are going to use the MNIST data set.
And we're going to do some user data
preprocessing like data normalization,
shuffling, and patching.
And for the model itself, it's going
to be very simple sequential model with three layers.
So the data set and model are ready.
And you need to instrument your code
to actually use the profiler.
EWA MATEJSKA: So since you're instrumenting the code,
does that affect your runtime performance?
JIHO CHOI: No.
We measure the performance overhead.
And it is very minimal.
EWA MATEJSKA: Oh, fantastic.
JIHO CHOI: So, for this demo, we're
going to use the TensorBoard Keras callback.
And you can check out the other modes
in the menu we are just probably seeing at the TensorFlow web
site.
EWA MATEJSKA: I see you have sampling mode on the screen
there.
What is that?
JIHO CHOI: Oh, yeah.
So sampling mode is the-- it enables on-demand profiling.
So for example, for already running jobs,
you can attach to it and then monitor the performance.
It is very useful, for example, for the products and jobs.
EWA MATEJSKA: What is the effect on your runtime performance
when you turn on sampling mode?
Same question, in context.
JIHO CHOI: Again, it's the performance are all great.
It's really minimal.
EWA MATEJSKA: OK.
Excellent.
JIHO CHOI: So this is how you add Keras TensorBoard callback
to use the profiler.
We are creating the TensorBoard callback object and then pass
it to the model fit function.
EWA MATEJSKA: Does that mean I'll
be then able to see the data in the TensorBoard?
JIHO CHOI: Yes.
Yes.
EWA MATEJSKA: OK.
Excellent.
JIHO CHOI: The profiler is available
as the dynamic plug-in to the TensorBoard.
So let's try running the model.
Here, I prepare the same code as I showed in the presentation.
And we can try running it.
And for the demo, I am only running
this model for five epochs.
EWA MATEJSKA: So we don't have to wait too long.
JIHO CHOI: Right.
EWA MATEJSKA: Yes.
JIHO CHOI: Then we need to start the TensorBoard, the instance,
to actually-- to look at the profile.
EWA MATEJSKA: OK.
And you do that by-- how do you start the TensorBoard instance?
JIHO CHOI: So you need to--
I already installed TensorBoard in my laptop.
Or you can-- the guide in the TensorFlow web
site shows the instructions how to download and install
the TensorBoard.
So now, I started the TensorBoard instance locally.
So let's go there.
EWA MATEJSKA: Oh.
And this is the data?
JIHO CHOI: Yeah.
This is some other data from TensorBoard.
And to see the profiler, we need to go to the profiler plug-in.
And this takes you to the overview page.
And it shows the kind of summary of the performance
of your model.
It shows the average step time and the step time breakdown.
And also, it provides some suggestions for the next step.
And it is over-- in this case, it says,
your model is highly input-bound.
And more than 40% of the step time
is spent on processing input.
And also, the overview page recommends you
to look at these two tools--
Input Pipeline Analyzer and Trace Viewer,
to find exactly what is going on.
EWA MATEJSKA: It's very actionable.
Awesome.
JIHO CHOI: Yeah.
So let's actually go to the Input Pipeline Analyzer.
And here, you can see that the input processing
time is taking a lot on the data preprocessing.
And that this tool also tells you
how to optimize this problem, and points
to different resources.
EWA MATEJSKA: Is the Input Pipeline Analyzer
part of the profiling tool?
JIHO CHOI: Yes.
It is one of the tools that we are providing
inside that the profiler.
So now we know that the input pipeline is the problem.
Then we can go some other tool provided
by profiler, which is Trace Viewer, which
shows you the very low-level, the trace of your model
execution.
EWA MATEJSKA: And this is because we added the sampling
callbacks in the data?
JIHO CHOI: Yeah.
This is the collected because we added that TensorBoard Keras
callback.
So this tool points you to the different resources
on how to optimize your model.
And then following the suggestions,
we can optimize our data preprocessing.
And there are a few things we can do.
What I did here is I used the parallel map instead
of the sequential map.
And I also cast the data set.
And then I also used the prefetching
at the end of the data preprocessing.
So I prepared the optimized code here.
And let's run the code.
Now again, we are only running for five epochs.
Then we can go back to our TensorBoard instance.
And you need to reload the page to get
the data I just collected.
So now you can see that the step time is reduced from 2.7
millisecond to 2.1 millisecond.
And the overview pages says, now your program
is moderately input-bound.
It's only 10% input-bound.
So that's a huge improvement.
EWA MATEJSKA: So it's a significant improvement.
JIHO CHOI: Yeah.
EWA MATEJSKA: Yeah.
So it's nice that you can compare the data
in between the runs in this way.
JIHO CHOI: Yeah.
Everything you collected are going to show up
in this dropbox menu.
So you can compare your new run with the previous runs.
EWA MATEJSKA: Awesome.
Thank you for showing us the TF Profiler tool.
Where can I learn a little bit more about it?
JIHO CHOI: Yeah.
So this demo is available as collab in the first link.
And we're also publishing the guide
in the TensorFlow website.
And there are some related talks to the profiler in the Dev
Summit you can check.
EWA MATEJSKA: Thank you, very exciting.
And thank you for joining us.
And the links will be posted separately.
[MUSIC PLAYING]