Name: P-ハッキング。クラッシュコースの統計＃30 (P-Hacking: Crash Course Statistics #30)
Uploaded: 2021-01-14T10:28:39.000Z
Duration: 11 min 2 s
Description: VoiceTubeの動画で発音を聞きながら英語表現を覚えよう！学べる英語：

Hi, I'm Adriene Hill, and welcome back to Crash Course Statistics.

We've talked a lot about how p-values let us know something significant in our data--but

those p-values and the data behind them can be manipulated.

P-hacking is manipulating data or analyses to artificially get significant p-values.

Today we're going to take a break from learning new statistical models, and instead look at

To recap to calculate a p-value, we look at the Null Hypothesis--which is the idea that

This can be no effect of shoe color on the number of steps you walked today, or no effect

of grams of fat in your diet on energy levels.

Whatever it is, we set this hypothesis up just so that we can try to shoot it down.

In the NHST framework we either reject, or fail to reject the null.

This binary decision process leads us to 4 possible scenarios:

The null is true and we correctly fail to reject it

The null is true but we incorrectly reject it.

The null is false and we correctly reject it.

The null is false and we incorrectly fail to reject it.

Out of these four options, scientists who expect to see a relationship are usually hoping

In NHST, failing to reject the null is a lack of any evidence, not evidence that nothing

So scientists and researchers are incentivised to find something significant.

Academic journals don't want to publish a result saying: “We don't have convincing

evidence that chocolate cures cancer but also we don't have convincing evidence that it doesn't".

In science, being able to publish your results is your ticket to job stability, a higher

In this quest to achieve positive results, sometimes things can go wrong.

P-hacking is when analyses are being chosen based on what makes the p-value significant,

Statistical tests that look normal on the surface may have been p-hacked.

And we should be careful when consuming or doing research so that we're not misled

It could come from a gap in a researcher's statistical knowledge, a well-intentioned

belief in a specific scientific theory, or just an honest mistake.

Regardless of what's behind p-hacking, it's a problem.

Much of scientific theory is based on p-values.

Ideally, we should choose which analyses we're going to do before we see the data.

And even then, we accept that sometimes we'll get a significant result even if there's

It's a risk we take when we use Null Hypothesis Significance Testing.

But we don't want researchers to intentionally create effects that look significant, even

When scientists p-hack, they're often putting out research results that just aren't real.

And the ramifications of these incorrect studies can be small--like convincing people that

eating chocolate will cause weight loss--to very, very serious--like contributing to a

study that convinced many people to stop vaccinating their kids.

For example, x-k-c-d had a comic associating jelly beans and acne.

So you grab a box of jelly beans and get experimenting.

It turns out that you get a p-value that's greater than 0.05.

Since your alpha cutoff is 0.05, you fail to reject the null that jelly beans are not

But the comic goes on there are different COLORS of jelly beans.

Maybe it's only one color that's linked with acne!

So you go off to the lab to test the twenty different colors.

And the green ones produce a significant p-value!

But before you run off to the newspapers to tell everyone to stop eating green jelly beans,

We know that there's a 5% chance of getting a p-value less than 0.05, even if no color

of jelly bean is actually linked to acne.

So what's the likelihood here that we'd incorrectly reject the null?

Turns out with 20 tests--it's way higher than 5%.

If jelly beans are not linked with acne, then each individual test has a 5% chance of being

significant, and a 95% chance of not being significant.

So the probability of having NONE of our 20 tests come up significant is 0.95 to the twentieth

That means that about 64% of the time, 1 or more of these test will be significant, just

by chance, even though jelly beans have no effect on acne.

And 64% is a lot higher than the 5% chance you may have been expecting.

This inflated Type I error rate is called the Family Wise Error rate.

When doing multiple related tests, or even multiple follow up comparisons on a significant

ANOVA test, Family Wise Error rates can go up quite a lot.

Which means that if the null is true, we're going to get a LOT more significant results

than our prescribed Type I error rate of 5% implies.

If you're a researcher who put a lot of heart, time, and effort into doing a study

similar to our jelly bean one, and you found a non-significant overall effect, that's

No one is likely to publish your non-results.

But we don't want to just keep running tests until we find something significant.

A Cornell food science lab was studying the effects of the price of a buffet on the amount

They set up a buffet and charged half the people full price, and gave the other half

The experiment tracked what people ate, how much they ate, and who they ate it with, and

The original hypothesis was that there is an effect of buffet price on the amount that

But after running their planned analysis, it turned out that there wasn't a statistically

字幕リスト動画再生

P-ハッキング。クラッシュコースの統計＃30 (P-Hacking: Crash Course Statistics #30)

significant

multiple

evidence

associate