at Google.

And today, we're going to talk about using audio

in the web platform.

The web platform has come a long way on both mobile and desktop.

And using technologies like the Media Capture API, Web Audio,

and WebRTC, it's now super easy to record, process, and send

audio and video in your app.

To demonstrate this, we're going to try

something we've never done before.

We're going to use this session to record a theme

song for Google I/O. And you're going to help.

So I'm pretty jazzed about this.

I played in some bands back in the day, a little keyboard,

a bit of bass.

And I remember what it was like to make music back then.

To get that perfect sound, we had

to tote around a whole bunch of gear, all sorts of effects

pedals-- overdrive, chorus, flanger, wah-wah-- amps and EQs

to get the right tone, and a four track to record it all.

And that was a big investment for us.

But that's what it took to sound like real rock stars.

So we'd all have been blown away to know

that this whole basement of audio processing gear

would soon be able to be replaced by a web browser.

In the modern web platform, we now

have the power to do incredible things

with media on any device.

And one great example of this is a new third party

app called Soundtrap.

Built on Web Audio, WebRTC, and Dart,

this app is like a set of instruments, a recording

studio, and a musical collaboration tool

all rolled into one.

And we're going to use it to record our song.

So musical inspiration can strike you at any time.

So imagine I'm just sitting on the train with my Nexus 7,

and I get the idea for a great song.

So let me show you how this works.

I'll plug in the Nexus 7.

Let's make this real.

So I'm running the Soundtrap app in Chrome for Android.

And the first thing we need is a driving percussion track.

So let me go ahead and check out the drum loops we have here

and figure out what's the right thing for our song.

[BEAT PLAYING]

Mm, let's see what else.

[BEAT PLAYING]

I think I want something that has a bit more rock.

[BEAT PLAYING]

Yeah, OK, that's the one.

So let me just tap that and add that to our song-- perfect.

OK, so we've got our percussion track.

But now, to complete our rhythm section,

we need to get a bass line going.

So fortunately, I've just arrived home,

where I've got my bass.

And now, using the Media Capture API, also known

as getUserMedia, I can record my instrument right

inside my laptop.

So let me show you how this works.

I'm going to plug in here just using a standard cable

right into the microphone jack on my laptop--

no special hardware or cables.

And there's my drum track I already added from before.

All right, I got a sweet bass amp.

[BASS PLAYING]

Let me just make sure I'm in tune here.

[BASS TUNING]

All right, perfect.

All right, let me just play a couple notes to get started.

This one might sound kind of familiar,

although our attorneys have told us

that it can't sound too familiar.

[BASS PLAYING]

All right, how does that sound?

All right.

[APPLAUSE]

This is sounding great.

And this is simply incredible to have

this kind of power in a web application.

So let's go ahead and record a bass line for our song.

So right here in Soundtrap, I'm just going to start recording.

And I'm going to add this bass line

right on top of the drum part that we had before.

[BASS PLAYING OVER DRUMS]

All right, I was a little bit early on that.

So let me try that one more time.

[BASS PLAYING OVER DRUMS]

All right, so there we've got our great bass line.

Let's save that out.

And that's what we've got for our bass track.

So onto our next part-- let's talk

about the technology behind what we just did.

So in the Media Capture API, the key method

is called getUserMedia.

And this call opens up the microphone-- oops.

This call opens up the microphone, asking the user

for permission the first time, and gives us a Media Stream

object that represents the audio coming from my bass.

Now, the same getUserMedia API can also

be used to get access to the microphone for speech

or a webcam for video.

Now normally, when we call getUserMedia to get access

to the mic, the mic signal is going

to get routed through a bunch of processing stages

to make sure we get a good speech signal.

Now, we're also going to apply auto gain

control to make sure the levels are correct,

as well as acoustic echo cancellation

to make sure any of the sound coming out from the speakers

doesn't also get picked up by the mic.

However, when we're recording an instrument,

we don't want any of that processing

to occur so we get as pure of a tone as possible.

So when we call getUserMedia, we're going to turn that off.

So here's a typical call to getUserMedia.

We make it on the Navigator object.

And we ask, in this case, for a media stream

that contains audio from the mic.

We'll get our stream back asynchronously.

And then, we can actually do something with it.

If the syntax looks kind of unfamiliar here,

this is making use of Dart's Future concept.

The .then at the end of the statement allows us to say

easily what should get executed when we get our stream back.

So when we call getUserMedia, we're

now going to specify the parameters

we want to the audio stream.

And here, we're going to ask for audio

and specify we want the AEC turned off-- simple.

So this way, it's what we're doing inside of Soundtrap

to get access to the microphone to record bass, guitar,

or any other instrument.

And now, let's turn it up a little bit.

Because Media Capture is really cool.

But it totally comes to life when

we connect it with a Web Audio API.

And that's where we can do some amazing audio processing.

So to demonstrate this, let me welcome Per on stage.

Per is one of the co-founders of Soundtrap.

[APPLAUSE]

Per.

Is one of the co-founders of Soundtrap,

and also a bit of a guitar hero.

PER: Hi, Justin.

JUSTIN: Great to have you here on stage, Per.

PER: Thank you very much for having us.

JUSTIN: So do you think you could play a couple guitar

riffs for us?

PER: Maybe I can do that.

I have a guitar [INAUDIBLE], so let me mic up.

JUSTIN: So Per is going to play some guitar for us.

And we're going to use that to demonstrate the effects that

are possible using Web Audio.

PER: So we're using Web Audio to build up

a whole set of effects.

And I'll show you some of the presets

that we have if I get some volume, yeah?

So this is just an acoustic sound with no effects

on at all.

[GUITAR PLAYING]

Like-- and then we can do something else.

JUSTIN: All right, let's go a little harder.

[DISTORTED GUITAR PLAYING]

We call that one--

[APPLAUSE]

We call that one "Fumes on the Ocean."

So let's kick it into overdrive.

[OVERDRIVE GUITAR PLAYING]

We call that one "Insane Bus Ride."

Is that right?

PER: Yeah, something like that.

JUSTIN: OK, all right, now let's take it back

to the origins of rock and roll.

[SLAP DELAYED GUITAR PLAYING]

OK, ladies and gentlemen, Per.

[APPLAUSE]

So to have this kind of processing power

on the web-- simply fantastic.

Let's talk about the magic behind these effects.

PER: Yeah, so let's talk about this simple one, actually.

So this is called a slap back effect.

[SLAP DELAYED GUITAR PLAYING]

Exactly, that's right.

JUSTIN: So most of you are probably

familiar with the HTML5 audio and video elements that

load, decode, and play out audio and video.

But Web Audio goes beyond that and gives you

super low delay and precise control

over each one of these steps, including

scheduling sounds down to sampling level accuracy.

And this time is particularly important for games, as well as

pro audio apps like Soundtrap.

Web Audio also allows powerful synthesis,

processing, and routing tools.

So this lets us do all sorts of great stuff, everything

you see here-- synthesizers, effects, visualizers--

everything you need for pro audio.

So from the API perspective, Web Audio

is based on a simple concept-- the audio pipeline.

And in the API, we refer to this pipeline as an AudioContext.

We can build out our pipeline by taking inputs either

from a file, microphone, or synthesizer,

and processing it using what we call nodes.

And these nodes can do various different things.

They can apply gain.

They can apply delay.

They can even send the data to JavaScript,

where JavaScript can change the sample values.

Finally, we send it all out to a destination--

typically the audio output.

Now I'm going to hand it over to Per

to explain how we get from this to this.

PER: Yeah, so now I'm going to show you the slap back.

So that was a very simple effect.

So you can hear it here.

So there's a direct signal, and then

one small delayed signal after that.

[SLAP DELAYED GUITAR PLAYING]

OK, you can hear that?

So that can be represented in a diagram like this.

So we have the audio coming from the guitar,

and then routed directly to the speaker.

That's the up line.

And then, we're adding one delay signal.

And then we gain that.

We lower the volume a little bit on that one.

So then you will have that very simple effect, yeah?

So the good thing with Web Audio is

that it makes it very easy to actually program

effects like this.

Because you have all the different types of nodes

you would like to combine.

And you can do that in very simple ways.

So I will just show you the code here

that you can use to implement [INAUDIBLE] like this.

So first, we need to get the context.

We talked about that.

So you're getting the context.

And you will use that object to, for example,

create the nodes later on.

And then we're also getting the user media.

So that was what we just showed before.

So once we get the user media back,

we'll start by actually creating the guitar node here

by using the Context object, and then just