Name: TensorFlowの内部。リソースとバリアント (Inside TensorFlow: Resources and Variants)
Uploaded: 2021-01-14T10:34:40.000Z
Duration: 46 min 55 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

And I'm here to tell you today about resources and variance.

And really this is a talk about state in TensorFlow

and stuff that got accidentally represented in state

I would love to be able to stand here, or rather

sit here, and tell you that an operation is stateful

if either executing it has a side effect,

But this is not what TensorFlow means by state flow.

Sadly, TensorFlow goes by the [INAUDIBLE] notion

that the meaning of a word is defined by its usage.

by this one bit that gets flipped and means all sorts

tf dataset from tensor slices has no side effects,

because the data set operations are value types,

And yet, that kernel is marked as stateful.

Because one of the effects of marking something

And constant folding can be buggy with data sets.

Iterators, on the other hand, are stateful.

But there are also some things in TensorFlow

So to differentiate while loops, we have stacks.

So that when you're doing the forward press of the loop,

So you can look at intermediate activations and stuff.

Tensor lists that you can use to aggregate stuff

from many iterations of a loop into a single view,

or do the reverse, they're also stateful in tf V1 and stateless

AUDIENCE: Is that because we didn't invent the stateless way

ALEXANDRE PASSOS: Because we did not invent the stateless way

So I want to spend the rest of the stock talking about how

statefulness is represented in tf V1, some of the problems

with that, how we're fixing those problems in tf V2,

and how we can deal with state, and also with things

that are not necessarily easily representable

the most obvious way is that if you will go on the TensorFlow

source code, and you find where ops are registered,

And the definition of state in TensorFlow

is that opdevs that have this bit set are stateful.

are going to look for that bit and behave differently

And this is something we need to clean up.

And I think we might have a chance to clean this up

with the MLIR dialect of TensorFlow, which

is going to have more finer grained bits.

But until then, we're stuck with this one bit

So among other things, what does this bit mean?

It means that TensorFlow will not do constant folding.

This includes the two or three separate systems

All of them know how to bypass stateful operations.

Similarly, there are at least two different places

in TensorFlow that do common sub expression elimination.

And they refuse to do common sub expression elimination

of stateful operations, which is very good, because if you're

to do that, and you have a neural network

with many layers, and your layers are initialized

from a random op, all of the layers with the same shape

would be initialized with exactly the same random values.

AUDIENCE: And all your prints would potentially

ALEXANDRE PASSOS: Only prints of the identical string

Because otherwise we would have enough information

But statefulness also means some things that

are not very obvious at all, like the op kernel

instances that the runtime uses to represent the computation

to run are reused across sessions for op kernels

字幕リスト動画再生

TensorFlowの内部。リソースとバリアント (Inside TensorFlow: Resources and Variants)

bunch

constant

guarantee

arbitrary