Placeholder Image

字幕表 動画を再生する

  • This video was filmed without sound.

  • Is it possible to use only these images to

  • reconstruct the sound? that is can you hear pictures?

  • in this video I'm gonna

  • try to demonstrate that it's possible to get sound from pictures but it's not

  • gonna be easy so I'm gonna need some help.

  • this episode was sponsored by

  • LastPass which allowed me to fly to the Bay Area to meet with one of my science

  • heroes so now on with the video.

  • How're you doing?

  • I'm sorry my place of my concern is like threw all

  • this crap over here.

  • So this is the experiment that I came up with. -It's like

  • a crumpled up ball of tinfoil? -yeah like if I had a more powerful camera and I

  • had like the right lens then you know we could do something that looks more like

  • you're spying on somebody. But we should be able to demonstrate that you can

  • recover like a rhythm or a sound from you know whatever camera you have.

  • Now you might think it would be easy to record sound in video because after all

  • sound is just vibrations so the air is vibrating back and forth and everything

  • it hits should vibrate back and forth too. So you'd think all we need to do is

  • video that motion and plot displacement versus time and then recover the sound.

  • But it's not that simple because for one thing I mean these sound vibrations are

  • incredibly tiny. They move objects only about one micrometer and even if you're

  • super zoomed in that is way less than a pixel we're talking a hundredth or a

  • thousandth of a pixel. -We're not seeing something that is at one pixel move to

  • an adjacent pixel. You're seeing one pixel get slightly darker and the

  • next pixel get slightly brighter. -What objects work best for recovering sound? -So

  • the things that work best are things that have a lot of damping but are also

  • very light so that they move very readily with changes in the air pressure

  • so what are some good examples? -well like a bag of chips.

  • you know the initial

  • experiments were very like contrived in a way. You know we had these objects on

  • optical benches we were blasting light at them

  • the sounds were like as loud as we could make them.

  • Mary had a little lamb, little lamb, little lamb

  • I figured we'd try to do a rhythm.

  • This is...

  • shave and a haircut

  • let's put that camera on a tripod

  • - oh yeah that's I mean that'd be great

  • All right, let's give it a shot,

  • this is the actual clip I recorded and I want you to

  • notice two things: first, you can't really see much motion; and second, there are

  • plenty of pixels getting dimmer and brighter because of image noise. I mean

  • it's not a pristine, perfect image so how do you tell the difference between

  • pixels getting brighter and dimmer due to tiny movements versus noise?

  • Essentially you look for edges in the image and then you say well if the object

  • moves by some fraction of a fraction of a pixel in one particular direction

  • then pixels on one side of that edge will get a little bit brighter pixels

  • the other side of that edge will get a little bit darker. And so basically what

  • we do is we sum together all the ones that are supposed to get brighter and

  • subtract all the ones that are supposed to get darker and then that gives us sort of

  • one number right and if you track that number over time then that gives you an

  • estimate of the displacement over time.

  • This is time but in samples and

  • then this is position.

  • -hmm so what do you do now?

  • well we're gonna try to do some

  • filtering on that.

  • it's clipping. You don't say

  • I mean it's not much... it's not much but you can

  • recognize two of the beats.

  • This is what we've recovered from a hundred and

  • eighty frames per second, which isn't really a whole lot within the range of

  • like audible frequencies so that's why you know kind of the most we can hope

  • for here is a bit of a rhythm

  • Now of course the main limiting factor

  • is framerate because we can hear sounds from 20 Hertz to 20,000 Hertz but most

  • cameras only shoot 30 frames per second so they miss virtually all of these sound frequencies.

  • Imagine this is the motion created by a 30 Hertz sound if

  • you try to capture this with a camera at 30 frames per second you would end up

  • seeing the object always in the same position because it's at the same point

  • in the wave cycle. So your perception would be that the object is not moving at all

  • So in order to measure a frequency of sound you actually need to

  • sample at least twice that frequency which is why a lot of music is sampled

  • at 44 or 48 kilohertz - that's more than twice as much as the highest sound we can hear.

  • At any rate if you want to get a something more intelligible you're

  • going to need some higher frame rate camera.

  • so we just went to the camera store and picked up a new camera that should be

  • able to shoot a thousand frames per second or thereabouts.

  • Is that gonna be enough?

  • it'll be enough for something.

  • Hehehe I love that confidence

  • This is one modulation away from dubstep right here.

  • Just a little bit more of a wump wump

  • and then we have the next big track.

  • I've set it to a thousand frames per second

  • now we're talking -yeah

  • okay

  • okay so you have the footage there and

  • you're cropping in a bit, tell me about that.

  • well we're running this on my

  • laptop as opposed to the servers that I had back at MIT and that is gonna mean

  • that if we run it on the full video of my laptop will crash -okay

  • so we're gonna crop it and try it on that - I can see a little bit of motion

  • -yeah I mean I think

  • that one question is whether that's like -resonant motion

  • yeah well in this case

  • would be kind of like the equivalent of a rocking chair like if the if the foil

  • has a like a rocking mode then that's actually not gonna give us a lot of

  • sound information -mm-hmm

  • Can you tell whether this is gonna work or not?

  • I am optimistic.

  • I think because I know what I'm listening for I can hear it in there but

  • yeah you've gotta be careful though really?

  • that you're well just gonna be careful

  • that you're not like confirmation bias sure.

  • let's try that

  • that's about 60 Hertz that seems a little much

  • okay we're gonna try one more time. We have basically put the piece of foil on

  • top of the speaker, we're dialing up the volume to... 11. -Well I mean it's a shower

  • speaker so

  • Cool So what do you think of that image there?

  • it's beautiful, it's gorgeous.

  • it has been like a couple years since I've looked at this code so I suspect I

  • might have forgotten something and I'm using it wrong but I want to point

  • something out here, which is that here's what we recovered and here is here is

  • the piano roll of the signal that we sent.

  • This looks like duh duh to me and

  • that's the duhhh yeah yeah

  • that's it that is the...

  • ---dih duh dud din duhhhh din dih - yeah okay so let's

  • see if we can actually get it to hear that

  • -We can see it, we just can't hear it

  • I think I know what it is.

  • What is it?

  • I don't think my laptop can play those frequencies.

  • hold on a second let me get some headphones

  • We need real speakers or something.

  • What do you hear Abe?

  • hold on.

  • I hear shave and a haircut two bits

  • here, listen this.

  • All right. -here, I'll hold it.

  • thank you.

  • yeah.

  • and... pitch shift

  • [recovered sound of Shave and a haircut, two bits]

  • yep there you go, visual microphone. [laughter]

  • this was a basic proof

  • of concept but Abe showed that with more powerful equipment he could recover

  • human speech from outside soundproof glass.

  • Have you ever considered that your

  • computer is a physical system that gives off vibrations each key on the keyboard

  • produces a unique sound due to its unique location. In fact, research has

  • shown that audio recordings of typing reveal 96% of keystrokes accurately.

  • Now this portion of the video was sponsored by LastPass, an app that stores all your

  • usernames and passwords so you never have to type them in. And this prevents

  • people from stealing your passwords by say recording audio of your keystrokes

  • or just looking over your shoulder.

  • What's even better is the convenience of

  • never having to fill in usernames and passwords or getting locked out again

  • With LastPass you don't have to write remember or reset passwords.

  • You get unlimited password storage and free cross-device sync.

  • When you open an app

  • or site on your computer or on an iOS or Android device LastPass fills in your

  • username and password this saves you valuable brain space so put your

  • passwords on autopilot with LastPass.

  • Now something I particularly like is if you

  • upgrade to LastPass premium you get advanced multi-factor authentication and

  • it works like magic.

  • So click the link below to find out more about LastPass

  • and thanks to LastPass for sponsoring today's show

This video was filmed without sound.

字幕と単語

動画の操作 ここで「動画」の調整と「字幕」の表示を設定することができます

B1 中級

あなたは画像から音を復元することができますか? (Can You Recover Sound From Images?)

  • 0 0
    林宜悉 に公開 2021 年 01 月 14 日
動画の中の単語