Placeholder Image

字幕表 動画を再生する

  • SPEAKER: It's a pleasure and an honor

  • to welcome our next speakers, Paige Bailey and Brennan Saeta,

  • who are going to talk to us about Swift for TensorFlow,

  • a next generation machine learning platform

  • that leverages innovation.

  • So please help me in welcoming Paige and Brennan.

  • [APPLAUSE]

  • PAIGE BAILEY: Good morning, everyone.

  • So thank you for coming to join us during the lunchtime.

  • We very much appreciate you-- you

  • taking the time to come to TensorFlow world

  • and also to learn a little bit more about Swift.

  • I'm Paige.

  • I'm the product manager for Swift for TF and--

  • BRENNAN SAETA: I'm Brennan.

  • I'm tech lead manager for Swift for TensorFlow.

  • PAIGE BAILEY: Excellent.

  • And so, first, we want to get a feel for the room.

  • How many of you have used Swift before?

  • Excellent.

  • BRENNAN SAETA: Awesome.

  • PAIGE BAILEY: And how many of you

  • attended Paris and Tim and their session, their workshop on--

  • earlier this week?

  • Few hands, cool.

  • So this is a great place to get started if you've never

  • used Swift before.

  • And it's also a wonderful place to learn

  • how Swift is getting more and more functionality

  • added specifically for machine learning and for deep learning.

  • So it's an exciting time.

  • Do you want to get started, Brennan?

  • BRENNAN SAETA: Yeah, let's do it.

  • PAIGE BAILEY: Perfect.

  • BRENNAN SAETA: So today we want to bring you a little bit

  • different kind of a presentation since a lot of you

  • actually know Swift, at least to some degree.

  • We wanted to really dig one level deeper and really

  • explore Swift for TensorFlow from its foundations.

  • So before we get into the meat of the talk,

  • I wanted to take a moment to acknowledge

  • a few very important folks.

  • So number one is--

  • this work would not be possible without the rest

  • of the Swift for TensorFlow team that really

  • does a huge amount of work to make this platform awesome,

  • so huge thank you to them.

  • But also not just the core team itself,

  • But there are a number of other teams

  • that we collaborate with at Google--

  • from the JAX team to the rest of TensorFlow and beyond.

  • And so we really are collaborating together

  • and building on each other's work to make something awesome.

  • So thank you to everyone, both at Google

  • and then, of course, our awesome open source contributors

  • beyond.

  • We have a lot of contributors that

  • are really making a huge difference to the project.

  • And, finally, for those of you who are familiar with Fast.ai,

  • some of the slides may be familiar.

  • So Jeremy Howard and Chris Lattner

  • co-presented a little bit Swift for TensorFlow

  • earlier this year.

  • And I encourage you all to check that out.

  • So, to begin with, I like to start with a question--

  • why Swift for TensorFlow?

  • PAIGE BAILEY: And I could give my answer.

  • BRENNAN SAETA: Let's hear it.

  • PAIGE BAILEY: Awesome.

  • So I've been using Python for a long time.

  • I really love the flexibility and often the user experience

  • that you get using the language.

  • It feels very understandable.

  • You read the code on the page and it reads almost

  • like a plain sentence and it also

  • has a vibrant ecosystem of data science tooling.

  • But something that I was always really frustrated

  • about whenever I was trying to deploy

  • Python models into production is that often you

  • have to do a lot of rewrites.

  • Part of the-- part of Python's benefits

  • don't necessarily translate so well to deployment scenarios.

  • So Swift is, to me, at least, a happy way

  • to get a lot of the great benefits of Python,

  • but also the benefits of a typed language.

  • BRENNAN SAETA: That's right.

  • And so what we are going for with Swift for TensorFlow

  • is an infinitely hackable platform

  • that gets you all the way from research, into production,

  • and, most importantly, that everything is customizable.

  • Because although there's a lot of flexibility

  • that you need to do research, you

  • need just the same amount of flexibility and capability,

  • often sometimes slightly different,

  • to really make a high quality production application.

  • PAIGE BAILEY: Yeah.

  • BRENNAN SAETA: And that's what we're

  • really excited about with Swift for TensorFlow,

  • and especially with the Swift platform,

  • how we can do interesting things to optimize our productivity.

  • So let's take a moment to ask the question what is Swift?

  • PAIGE BAILEY: And Swift is a programming language

  • that was created by Chris Lattner pretty recently.

  • BRENNAN SAETA: Yeah.

  • PAIGE BAILEY: One, he was spending some time

  • over at Apple.

  • Since then, Chris has come to join us and worked at Google.

  • You'll be hearing a lot about one of Chris's other projects

  • later this week.

  • MLIR, I believe, is in the keynote tomorrow.

  • But Swift is a programming language

  • that's focused on that extensibility component

  • that Brennan just mentioned as well as readability, usability,

  • and production readiness.

  • BRENNAN SAETA: That's right.

  • So this is Swift for TensorFlow from foundation.

  • So let's really dig in and let's really go from the foundations.

  • So I think an important question to understand

  • what is Swift at a deeper level is to understand

  • what is a compiler.

  • And so as you program computers, there's

  • two fundamental things that are immovable

  • and that like to think about the world

  • in their own particular ways that

  • are sometimes incompatible.

  • You've got humans, on one hand, that

  • want to think about the world in a particular way

  • and you've got hardware which also

  • thinks about the world in its own particular way.

  • And a programming language is basically an intermediate point

  • in between these two immovable, fundamental truths

  • of technology.

  • And so programming languages are designed for humans

  • to express ideas, but they're also designed

  • to be executed on hardware.

  • And they're different points in that space that you can pick

  • and Swift is just one of such points.

  • But we think it's a really interesting one

  • and we'll talk about why in a couple of minutes.

  • PAIGE BAILEY: Right, so one of the benefits of having

  • a typed language is that a lot of the responsibility that

  • might be pushed on you if you're using a language like Python

  • is suddenly abstracted away.

  • So I don't have to care so much about baking performance

  • optimizations into my code immediately,

  • the compiler can just do it for me.

  • Another great thing is that if you look at the screen

  • right now, you can see an example

  • of a few commands in Swift.

  • It looks very, very similar to Python.

  • There's some lets sprinkled in, there

  • are some-- a few indications of types, but other than that,

  • pretty straightforward to understand.

  • If you're only experienced using typed languages as something

  • like C++, this probably looks a great deal friendlier.

  • BRENNAN SAETA: That's right.

  • And Swift is a very modern and powerful programming language.

  • So even though Swift is statically typed,

  • you don't actually see very many types in advanced Swift code.

  • Here we're defining a dictionary numbers

  • from strings to lists of integers,

  • the compiler can just figure this out for you.

  • One of the principles of Swift is that you

  • have a very helpful compiler.

  • And you can define higher order operations,

  • reduces a common pattern from functional

  • programming across a sequence of elements.

  • And you can then call this x.sum,

  • mapped over all the values in the dictionary.

  • And voila, you've now used a number of higher order

  • functions together without having

  • to write types in a very seamless and nice manner.

  • So what we're trying to get across

  • is that Swift is almost as high level as Python.

  • So this is actually a slide that I am borrowing from Chris

  • and Jeremy's Fast AI lectures.

  • And at the top, you have for Swift for TensorFlow model.

  • And at the bottom, you actually have a Python model.

  • And if you look at the code, if you squint a little bit,

  • they're almost exactly the same thing, right?

  • You've got a few more self dots on the bottom.

  • And you've got some more curly braces on top.

  • But they're basically the same thing.

  • So Swift is just as high level, almost, as Python.

  • PAIGE BAILEY: We also have fewer lines of code in that example,

  • is that right?

  • BRENNAN SAETA: Well, it depends on how

  • you count the curly braces.

  • So little bit of here, little bit of there.

  • But what's really important about Swift is that it's

  • designed to be fast, right?

  • It's practically in the name.

  • So who here has actually looked at Assembly in the last six

  • months?

  • Ah, fewer hands this time.

  • Well, bet you didn't expect to be looking at Assembly today.

  • So for those of you who don't know,

  • the Godbolt Compiler Explorer is really neat.

  • You can plug-in some source code on the left-hand side.

  • And it will give you the assembly

  • on the right-hand side.

  • And here we have Swift.

  • We're calling a higher order function reduce on a generic

  • type-- so it's actually an array, in this case,

  • an array of integers--

  • passing in the plus function, reducing over everything.

  • And if you look at the loop that is going on here,

  • so you've got your function prelude.

  • You do a couple checks to make sure

  • that everything is reasonable.

  • You actually just have these five instructions

  • that form the main body of your loop.

  • You do the add.

  • You check to make sure that you don't have any overflows.

  • You then increment one to get to the next value in the array.

  • You compare to make sure you've not

  • reached the end of the array.

  • And if you haven't, then you do your loop again.

  • This is like the most efficient set of instructions

  • you could possibly do to sum across an array.

  • And so even though you have all these higher order things,

  • you're not even writing types, the Swift compiler

  • can helpfully optimize it all the way down for you.

  • Now actually, for those of you who

  • really know about Performance and Assembly,

  • a really helpful compiler will actually

  • vectorize this operation for you.

  • And so Swift, by default, is safe.

  • So when you use Plus, you're getting safe integer

  • add that checks for overflows.

  • But if you do Ampersand plus-- the Ampersand plus function,

  • this says just give me--

  • 2 is compliment, don't bother checking for overflows--

  • and the Swift compiler will hopefully

  • vectorize your operation for you to make

  • it run extremely quickly.

  • So Swift is designed to be safe, to be high level,

  • but of course, also fast.

  • So programming languages are all about atoms and composition.

  • And so the atoms in Python are C code

  • that implements Python objects.

  • And the composition is an interpreter

  • that lets you combine these calls in very interesting ways,

  • right, like Python array, Python dictionary.

  • They're very, very carefully implemented in Python

  • to be as quick as possible in C. And then

  • you get to build your whole universe on top of it.

  • C++, however, takes a different tack.

  • There is no C underneath C++, right?

  • It's just C++.

  • So the atoms in C++ are int, float, C arrays, and pointers.

  • You got a few other atoms as well.

  • And you could develop your own structures in C++,

  • like structs classes, std::complex, st::string,

  • std::vector.