is talk about hypothesis testing,

which is the heart of all of inferential statistics,

statistics that allow us to make inferences about the world.

So, to give us the gist of this,

let's start with a tangible example.

Let's say, hypothetically,

you run a website that has the mission

of giving everyone on the planet a free education,

and you wanna think about how you might change

the amount of time people spend on the site.

Ideally, you wanna increase the amount of time

people spend on the site

so there's more learning on the planet.

Well, currently the website has a white background like this

and the mean amount of time people spend

when you have a white background,

the mean amount of time when you have a white background

is 20 minutes.

And you or someone on your team,

maybe you read some type of study that says

people like to spend more time on yellow backgrounds.

I don't actually think that's true,

but let's just go with that for the sake of this video.

And so you have a hypothesis

that if you actually have a yellow background,

if you change your background to yellow,

that the mean amount of time that people spend

on a yellow background, on yellow,

is going to be different, is not going to be equal to

the mean amount of time people spend on a white background.

So, the question is how do you test this,

and how do you feel good about your inferences

that you make from your test?

And that is the heart of hypothesis testing.

And medical research, actually almost all research

involves some form of hypothesis testing.

So, how would you do this?

Well, the standard why to do this

is to set up a couple of hypothesis.

Hypotheses, I should say.

The first one is known as your null hypothesis,

and I often think about this as the skeptic's hypothesis.

Skeptics think that,

hey, it's hard to make a difference in this world,

or cynics feel like it's hard

to make a difference in the world

and so they always have this null hypothesis

that's saying, "Hey, you think you're making a difference,

"but you aren't."

So, the null hypothesis is that

the mean amount of time people spend on the yellow site,

or on a yellow site,

is going to be equal to the mean amount of time

that people spend on the current site

or the existing site or on a white site,

while the people who are thinking about,

"Hey, how do I make change?

"How do I make improvements in the world?"

they had some type of hypothesis

and we call that the alternative hypothesis.

And so the alternative hypothesis, A for alternative,

is that the mean time on the yellow site,

on the yellow site,

is actually different.

Is actually different.

It is not equal to the mean amount of time

on the white site.

So, how do we think about this

now that we set up these hypotheses?

Well, what we're going to do is

we are going to assume,

we assume the null hypothesis.

Then we build this yellow site

and then we take a sample

of the people using the yellow site,

and we say, "What is the probability

"of getting that sample mean,"

which is an approximation of the parameter of the true mean,

"what is the probability of getting that sample mean

"if we assume the null hypothesis?"

And if the probability of getting that sample mean

on the yellow site,

assuming the null hypothesis, is really low,

then we reject the null hypothesis,

which suggests the alternative.

On the other hand, if we get a sample mean

that seems pretty reasonable to get

if you assume the null hypothesis,

then we fail to reject the null hypothesis

and then that would not suggest the alternative.

Now, to make this a little bit more tangible,

and we'll go over this into a lot of videos,

if you assume the null hypothesis,

then there's a few things you can think about.

You can think about just the general distribution

of the amount of time people spend on the site.

It would look something like this.

We will, for this sake,

assume that it's a normal distribution,

and normal distributions are very important,

and/or things that are close to normal distributions,

for hypothesis testing.

But let's say that it's a normal distribution

of the amount of time people spend on the site

and so there is some mean.

We know that mean,

so the mean that people spend on that white site

is equal to 20 minutes.

And, remember, we're assuming the null hypothesis,

so we're assuming that this is also the amount of time

that people would spend on the yellow site.

We've assumed, assuming, the null hypothesis,

and you could view this as time or distribution

of time spent.

Now, one of the things we're going to talk about

in future videos is if you have this distribution,

you can actually come up with another distribution

of the means of samples you might get.

So, there's something else called the sampling distribution,

and I know it's very confusing at first.

Sampling distribution of the sample

of the sample mean,

and it'll be for a given sample size,

for sample size, sample size.

Let's say this is sample size 1,000.

I'm just making things up.

I could've said N,

but I'm just gonna make this a little bit more tangible.

Well, we're going to get statistical methods

for how you can think about this distribution

assuming this distribution we have on the left.

And it turns out this distribution

is going to look like the one on the left,

but it's going to be narrower around that mean.

It's going to look something like this.

And, actually, the larger your sample sizes are going to be,

the narrower it's going to get.

Now, remember, this isn't just the distribution

of the amount of time people spend on the site.

This is the distribution that if I were to take a sample

of the amount of time people spend on the site

and calculate the means,

this is the distribution of those sample means I might get.

Now, the center of this distribution is still

our mean for white which is equal to the mean for yellow.

Remember, we're assuming the null hypothesis.

The mean for yellow.

But each of these points,

for example, if I think about this,

this is amount of time that someone might spend

and you can see that there's a low probability about it.

This over here, this would be a sample mean you might get

for a time that you sampled 1,000 people

and you calculated the mean,

and you see that there's a low probability for it.

So, then what you would do is,

if you were able to statistically generate these things

assuming the null hypothesis,

and don't worry too much,

we'll find out the techniques for doing this

and the assumptions we need to make for doing this,

what we do is then take a sample of 1,000.

So, you take your sample of 1,000,

so sample 1,000,

and then from that you are able to calculate a sample mean.

You are able to calculate that.

And let's say you get a sample mean of 30 minutes.

And let's say, actually, that that is right over here,

that this is 30 minutes right over here.

The center was 20 minutes.

The next thing, what you do is you say,

"What's the probability of getting a result

"at least that extreme assuming the null hypothesis?"

And that high probability on these curves,

it would be this right tail here

and it would be the left tail

that is equally far on the left side,

so it'd be like that.

And what you do is you look.

You look at this probability,

which would be these yellow areas there,

and then we think about the probability

of getting a result at least as extreme as 30 minutes.

So, probability of getting,

getting a sample mean at least

as extreme

as the sample mean equaling 30 minutes,

assuming, assuming

your null hypothesis,

and that's exactly what those yellow areas are all about.

And you compare that to some pre-specified threshold.

So, that threshold is oftentimes 5%.

Sometimes it's 1%.

But if this probability is less than or equal to,

if it's less than or equal to your threshold,

and the threshold is oftentimes denoted

by the Greek letter alpha,

well, we say, "Hey, that was a very low probability

"of getting a result at least this extreme

"if we assume the null hypothesis,"

and so that will allow us to reject,

reject the null hypothesis,

which would suggest, suggest the alternative.

Notice we haven't proven the alternative.

We also haven't proven the the null hypothesis

is for sure false.

We've just said if we assume the null hypothesis,

there's a very low probability of getting a result

at least as extreme as what we just got,

so we will reject the null.

Now, if it's the other way around,

if the probability of getting a sample mean

at least as extreme as this is still reasonable,

if it's greater than your pre-specified threshold,

then you fail to reject the null.

You fail to reject

your null hypothesis.

So, I'll leave you there.

In future videos,

we'll go into much more depth into all of this,

but this is to give you a sense of how hypothesis testing

allows science or all of us in the world

to start making inferences that we can feel good about.