Name: あなたは画像から音を復元することができますか？ (Can You Recover Sound From Images?)
Uploaded: 2021-01-14T10:25:29.000Z
Duration: 11 min 23 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

基礎受動態

様態の副詞

Is it possible to use only these images to

reconstruct the sound? that is can you hear pictures?

try to demonstrate that it's possible to get sound from pictures but it's not

gonna be easy so I'm gonna need some help.

LastPass which allowed me to fly to the Bay Area to meet with one of my science

I'm sorry my place of my concern is like threw all

So this is the experiment that I came up with. -It's like

a crumpled up ball of tinfoil? -yeah like if I had a more powerful camera and I

had like the right lens then you know we could do something that looks more like

you're spying on somebody. But we should be able to demonstrate that you can

recover like a rhythm or a sound from you know whatever camera you have.

Now you might think it would be easy to record sound in video because after all

sound is just vibrations so the air is vibrating back and forth and everything

it hits should vibrate back and forth too. So you'd think all we need to do is

video that motion and plot displacement versus time and then recover the sound.

But it's not that simple because for one thing I mean these sound vibrations are

incredibly tiny. They move objects only about one micrometer and even if you're

super zoomed in that is way less than a pixel we're talking a hundredth or a

thousandth of a pixel. -We're not seeing something that is at one pixel move to

an adjacent pixel. You're seeing one pixel get slightly darker and the

next pixel get slightly brighter. -What objects work best for recovering sound? -So

the things that work best are things that have a lot of damping but are also

very light so that they move very readily with changes in the air pressure

so what are some good examples?  -well like a bag of chips.

experiments were very like contrived in a way. You know we had these objects on

optical benches we were blasting light at them

the sounds were like as loud as we could make them.

Mary had a little lamb, little lamb, little lamb

this is the actual clip I recorded and I want you to

notice two things: first, you can't really see much motion; and second, there are

plenty of pixels getting dimmer and brighter because of image noise. I mean

it's not a pristine, perfect image so how do you tell the difference between

pixels getting brighter and dimmer due to tiny movements versus noise?

Essentially you look for edges in the image and then you say well if the object

moves by some fraction of a fraction of a pixel in one particular direction

then pixels on one side of that edge will get a little bit brighter pixels

the other side of that edge will get a little bit darker. And so basically what

we do is we sum together all the ones that are supposed to get brighter and

subtract all the ones that are supposed to get darker and then that gives us sort of

one number right and if you track that number over time then that gives you an

I mean it's not much... it's not much but you can

This is what we've recovered from a hundred and

eighty frames per second, which isn't really a whole lot within the range of

like audible frequencies so that's why you know kind of the most we can hope

is framerate because we can hear sounds from 20 Hertz to 20,000 Hertz but most

cameras only shoot 30 frames per second so they miss virtually all of these sound frequencies.

Imagine this is the motion created by a 30 Hertz sound if

you try to capture this with a camera at 30 frames per second you would end up

seeing the object always in the same position because it's at the same point

in the wave cycle. So your perception would be that the object is not moving at all

So in order to measure a frequency of sound you actually need to

sample at least twice that frequency which is why a lot of music is sampled

at 44 or 48 kilohertz - that's more than twice as much as the highest sound we can hear.

At any rate if you want to get a something more intelligible you're

going to need some higher frame rate camera.

so we just went to the camera store and picked up a new camera that should be

able to shoot a thousand frames per second or thereabouts.

This is one modulation away from dubstep right here.

I've set it to a thousand frames per second

you're cropping in a bit, tell me about that.

laptop as opposed to the servers that I had back at MIT and that is gonna mean

that if we run it on the full video of my laptop will crash -okay

so we're gonna crop it and try it on that - I can see a little bit of motion

that one question is whether that's like -resonant motion

would be kind of like the equivalent of a rocking chair like if the if the foil

has a like a rocking mode then that's actually not gonna give us a lot of