sciences.

Although a random selection process is generally the best way of getting a representative sample

from a population, it doesn’t guarantee a perfect sample.

We must acknowledge that even the best random samples will always be a little different

from the true population. We call that “sampling error”.

It occurs when we take a random sample rather than observe every subject in a population.

Let’s pretend that you are conducting a telephone survey on how much people spend

during their summer vacation. You call 1000 randomly selected U.S. households and just

by dumb luck after a 100 phone calls you happened to get a hold of Mark Zuckerberg, founder

of Facebook, and he agrees to take your survey. Unlikely, but possible.

Let’s also pretend that after calling 600 people you also got a hold of Oprah Winfrey.

Again, unlikely, but the point I’m trying to make is that if, just by random chance

or luck, we got slightly too many rich people in our sample, or too few wealthy people,

our sample will look a little different than the true population. That difference is called

sampling error.

When collecting a sample, we can’t avoid sampling error, but we can estimate the size

of sampling error and there are ways of reducing sampling error.

The margin of error that you commonly see with survey results is an estimate of sampling

error. Because it is just an estimate, there is a small chance, usually 5% or less, that

the margin of error is actually larger than stated in a report.

We can reduce sampling error by increasing the sample size, that is you can select more

subjects to observe. As your sample size increases, your sampling error decreases.

But increasing your sample size also increases costs, both in time and in money. And after

about a 1000 cases you start to get less bang for your buck.

As you can see in this chart, after a 1000 cases, even if you more than double your sample

size to 2500 subjects you only reduce your margin of error by 1%.

You can also reduce sampling error with a good sampling design. For example, if your

overall population has distinct subpopulations, then sampling each subpopulation independently

may reduce sampling error.

But these techniques can only reduce sampling error so far. The only way to remove sampling

error completely would be to observe every element in a population, which is impractical

if not, in some cases, impossible. We simply must acknowledge that survey samples are imperfect,

but generally a very efficient and accurate way of studying a large, complex population.