Siris on in this episode was specifically going to start looking at integrating sound into the emulation.

I'm going to continue the code exactly where we left off from the previous episodes.

If you've not watch those episodes, I recommend that you do so first.

But before we get started, it occurred to me that we've not yet integrated sound into any pixel game engine applications.

I have done a whole Siris on working with sound in my code it yourself synthesizer Siris on the Pixel game engine adopts a very similar approach, but I thought it would be useful first to create a small demonstration up just to show how the pixel game engine handle sound.

It might be useful things, both of them emulation.

So let's get stuff.

This is the code base that we've been working with and can be found on the get hope.

It's the Nez emulator so far, but you'll notice I've included a lot more mappers.

Now I'll talk specifically about additional mappers in a later video of this Siri's, but for now, it just gives us a bit more variation in the number of games that we contest, and I'm going to hijack this project temporarily just to build this sound demonstration video, someone's add a new file and I'll call it sound demo dot CPP on just temporarily.

I'm going to remove the nose emulator.

This is so we don't have to into remains in our project.

The first thing I'm going to do is pull in the demonstration code that comes in the pixel game engine Head of file.

This is the one that draws all of the randomly colored pixels to the screen.

I'm going to stop it from doing that.

Unlike the console game engine sound in the pixel game engine is facilitated.

Very an extension.

We call them Peg X.

We've seen these before with two D and three D graphics, so I'm just going to include the extension toe.

Activate the sound extension.

I'm going to initialize it in on user creates, and I'm setting it to 44 100 which is the sample rate of the audio system.

I want one channel, so it's really only mano The nest was a mano device on those of you that have seen my code it yourself synthesizer Siri's.

You will have seen these before, but these last two parameters tell the system how to package the data to send its to the sound hard work.

You can change them to change the performance of the real time nature of the audio, but I found that these two settings worked very well for most applications because the sound Peg IX opens a handle to some sound hard work.

We need to release that handle when the application is finished.

So this is one of those unusual situations where we're going to override the often under used on user destroy function of the pixel game engine.

This is called Whenever you close the window or you return false from on user update.

It gives you an opportunity to clean up after yourself, which you should do.

The sound extension works in the following way.

As is well established on this channel.

The pixel game engine creates a thread that it uses to call the on user update function on this that sits in a loop repeatedly running as fast as it can unless you've got the V sink option enabled when you constructed it.

The problem we face is that the speed off this main set conflict you ate depending upon what the function is doing on normally, we would alleviate this fluctuation by exploiting the F elapsed time variable that comes with it.

However, sound is unforgiving.

The human ear is very sensitive to changes in audio and cracks and pops and buzzes.

Which reminds me, actually, as this is a video about audio, uh, I will warn you Now we're going to have all sorts of squeals and scratches on interesting noises and probably inappropriate volumes throughout, So consider yourself one.

Anyway, the point I'm trying to make is we can't just issue sound samples to some audio device because the rate at which we issue the sound samples won't be consistent.

Instead, samples must be generated at the rate the audio device expects.

So toe handle this.

A second thread is maintained in the sound peg eggs on the speed at which this thread operates is largely governed by the sample rate that we specified when we created the instance of the peg X.

This way we can ensure that sound samples are delivered to the audio device hardware appropriately, but to get the samples that we need the Peg X is going to request from the main thread a sample of sound when it needs it on the main said will then return that single sample.

In effect, this request is going to interrupt our main threat.

Therefore, to use the sound extension, we need to be aware that there are actually multiple threads now in our application.

But this allows the sound hard work to run at the speed it needs to run out on.

This allows our application to run as fast as possible.

Now I'm quite sure they're going to be many more sophisticated and clever ways toe handle this interaction between the two threats, but I am aiming to keep things simple, so I'm going to create a function, and it's a static function called sound out.

This is the function that will be called by the sound extension every time it wants a single sample, and it requests what channel it wants that sample to before.

So in a two channel system that's the left speaker, followed by the right speaker.

It also provides some convenient timing functions.

Global time is the time since the application started and time step is effectively won over the sample rate that we specified earlier.

We need to tell the sound extension the name of this function, so we can call it when it needs a new sample on will do that, using the set to use a synth function.

There's actually several different over rideable functions that can exploit different properties off the sound extension.

But this function tells the sound extension.

Hate were generating audio samples, so please call this function periodically.

Now let's generate some sound good to keep it simple again on Just return a sine wave.

Tell me to take the sign off F global time.

That's effectively my ex in the time domain.

I'm going to multiply that by a frequency 440 hurts in this case, which I believe is an A, but we need to convert that frequency into on angular velocity.

Basically, we're converting it into radiance for the sine function, which we achieve by multiplying by two times pi.

It's all very approximate here, so let's take a look and let's hope that the volume is sensible.

Hopefully, in the background, you can hear the sine wave play.

I'm really going to regret making a video about audio this way because I'm simultaneously recording my voice on the desktop output so the two might not mix very well.

Generating a static sine wave is all very well and simple, but we know that we're going to want to do something a little bit more interactive and dynamic for our emulation.

So let's have a go at implementing a square pulse wave.

This is a very simple wave form on.

We've seen it before in the code it yourself since the size of Siri's.

It's a wave that looks like this in time.

There are many ways to generate this way form the most obvious one being to simply have a branching condition.

To say whether it's high or low is that the correct time.

But when you start adding such discreet components to an oscillator like this, it becomes hard to work with when you're doing things such as changing the frequency or changing the face or the more advanced things later on.

It can also lead to fairly ugly artifacts, because the sound hardware isn't capable of these entirely vertical transitions.

So I thought it would be interesting to look at creating a square pulse wave using sine waves.

After all, we can construct all sounds out of the addition off sine waves, a square pulse wave differs from a regular square wave.

In fact, what I've drawn here is a regular square wave.

It is low for 50% of its period, and it is high for 50% of its period.

A pulse wave allows you to change this ratio, and that's called the duty cycle, so we might have it high for a short period and low for the remainder.

Perhaps in this instance, it's high for 10% on low for 90% this ounce.

Texture to the sound on the synthesizer.

Aficionados out there will understand exactly what I mean, but it's very difficult to describe verbally.

You need to hear it in order to understand it.

So I think we'll create a function which allows us to specify a square pulse wave of any frequency amplitude on duty cycle.

Now, lots of memories flashing back here because I love Desmond's dot coms.

Graphing calculator.

It's great for making videos about wave forms on.

I use it a lot in the synthesizer, Siri's so here I'm drawing a regular sine wave on defended some sliders.

Here I can change the frequency of the sine wave as the frequency increases the pitch of the notes that we hear goes up.

Broadly speaking, you'll see that the sine wave is the one that we've just implemented.

But what I'm going to do is add together lots of sine waves at frequencies that are inter just steps from each other.

I'm going to call these harmonics on.

I can add more sine waves by changing this slider here.

And so as we add more sideways, we see we start to form a sore tooth wave.

That's very nice if I had a second sine wave exactly the same but change its phase.

So I'm going to make it 50% out of face with the original sine wave.

So when the red one is up, the purple one is down Very, very simple.

And again, this sine wave can be influenced by this slider toe.

Add more harmonics to it.

So I've got to soar waves.

That's a 50% out of phase with each other.

To generate a pulse wave, we simply subtract one from the other, which we can see here with the black wave, which is a very nice square wave in a continuous time domain at a 50% duty cycle, it is equally low on equally high on the quality of this square wave depends on how many harmonics are in our sore waves.

As you can see as we go to a low account, the school wave becomes rounder on dhe.

This also adds interesting flavors to the resulting sound.

When this way for miss played naturally, the more times that we want to add sine waves, the more computation is required.

So creating a really accurate square wave like this is actually quite computational intensive were performing these operations while almost 100 times in this instance.

But interestingly, because we've got two bass way forms that we can have out of phase with each other, we can change the duty cycle of our pulse wave.

It does sound a little bit of D C offset, but in audio, that's not very important.

Tell me the change that matters.

That's what we hear coming out of the speakers.

So I'll just disable these background wait forms on there.

We can see we've now got a way of calculating in continuous time a pulse wave function.

And if I increase the frequency that behaves to.

What's nice about this approach is that we end up with a way form which is applicable to the real world.

There is no vertical lines as part of this way form of various steep radiance.

But they're not vertical, which means in principle our sound hardware should have no problem replicating.

This sound mean there are some laws that govern whether it can or cannot.

But it does also mean that we can do lots more interesting mathematics to this way form if we choose to do so.

So let's implement this way.

Form into the pixel game engine now and have a listen to it on.

I'm going to be very quick and dirty with this.

I'm just going to add three atomic floats because I'm going to change the value of these floats in the pixel game engine fed.

But I need them to get through to the sound generation thread later, as I mentioned before, lots of Tidier ways to do this.

But this is quick and simple for a video on.

I'm also going to add on additional function called Sample square wave on will generate our pulse wave samples in this function, the parameters that it takes in our the frequency that we want to listen to and the time on I'll start by adding three variables.

A and B represents the sample values for the underlying sine wave forms.

And P is going to represent the phase difference between the two, since we're going to be summing up a whole bunch of sine waves depending on how many harmonics were interested in a lot of four loop, both sine waves get the same argument, so I'll cash that here in variable C.

It's the number of harmonics times the frequency times two times pi times t on as I've just shown in Dismas.

The two sine waves are very similar, except one is offset by the face, and I'm also scaling by the number of harmonics.

This keeps things within a reasonable envelope.

Once all of the Sinus sides have been summed, I'm going to return the subtraction between the two as our sample value I've scaled.

It's a little bit here simply because I obviously already pre implemented this algorithm.

I know what it's going to look and sound like on.

I want to scale it so I can visualize it on the screen as well as hear it.

So now when the sound engine requests a sample, instead of just returning a sine wave, I'm going to call our square way function.

With the correct parameters.

Square waves can get quite loud, so I'm going to change.

The amplitude of this one is effectively changing the volume of it in on user update.

I want to visualize the way form on the screen like we just saw in Desmond's on.

I'm not even going to talk through the code for this visualization.

It is totally irrelevant for this video, but you can go and have a look at it.

I'll make sure I upload this little demonstration file to the get hope.

It's full of magic numbers and Constance that's just to scale it appropriately on the screen.

And I'm also going to be sensitive to Cem user input.

So pressing the Q and A keys is going to change the duty cycle.

Pressing the D and E Keys is going to change the frequency on pressing the S and W Keys is going to change the number of harmonics that we're constructing our way form out off.

And finally, because we've created some static variables, we need to initialize those two.

So let's take a look.

We'll have a listen.

Well, can certainly hear what sounds like a traditional square way in the background and to reduce the number of harmonics on.

Eventually, it'll begun to sound just like a sine wave, as we had before.

In fact, if I press this slowly, we can hear the additional frequency is being added into the sound.

So I wrong speak well.

Just listen.

Harmonics.

It becomes more buzzy and gives us that classics and metro sounding loveliness that we all love.

It can change the duty cycle with the Q and A thief.