Grade by Grade
C. Organization of Data for Analysis · Organization of Data for Analysis

Variability in Data and Sampling Variability

C1.2About 65 minutes
Learning goals
  • I can explain why variability is part of almost all data.
  • I can identify natural, measurement and sampling variability in a situation.
  • I can explain why different random samples give different statistics, and how sample size affects this.
  • I can compare sample statistics with a population value and judge whether a difference could be due to chance.

Words to know

variability
The way data values differ from one another. Variability is why we need statistics at all.
natural variability
Differences that occur because individuals really are different (no two maple trees have exactly the same height).
measurement variability
Differences caused by the measuring process: the instrument, the person measuring, or the conditions.
sampling variability
The way a statistic (such as a mean or a percentage) changes from one random sample to another.
population
The entire group that a question is about.
sample
The part of the population from which data are actually collected.
parameter
A number that describes a whole population, such as the true percentage of all students who walk to school.
statistic
A number calculated from a sample, used to estimate a parameter.

No two snowflakes, no two samples

Measure ten maple leaves from the same tree and you will get ten different lengths. Weigh the same apple three times on a kitchen scale and you may get three slightly different readings. Ask two random groups of 30 students how they get to school and you will get two different percentages.

That is variability: data values differ. If every value were the same, you would need only one measurement and there would be no need for statistics. Statistics is the science of making sense of variability: describing it, measuring it, and deciding when a difference is real and when it is just chance.

Three sources of variability

  1. Natural variability. Individuals really are different. Students differ in height, sleep and opinions. Two bean plants grown in the same soil with the same water still grow to different heights because of genetics and tiny differences in their surroundings.
  2. Measurement variability. The measuring process adds its own small differences. A stopwatch started a fraction of a second late, a tape measure read at a slight angle, a survey question that people interpret in different ways.
  3. Sampling variability. When you study a sample instead of the whole population, the result depends on which individuals happened to be chosen. Another sample would give a slightly different answer.

Some textbooks add a fourth source, induced variability: differences caused by outside factors that were not controlled, such as one group of plants getting more sunlight. Good study design tries to remove it.

Naturalsame seeds, same care,different heightsMeasurement3 readings of one pencil:12.3 cm, 12.5 cm, 12.4 cmtool and person addsmall errorsSamplingpopulationsample 130%sample 240%two random samples,different results
Three sources of variability. Natural differences are real; measurement and sampling add their own differences on top.

Parameters and statistics

A parameter describes a population. A statistic describes a sample. We usually cannot know the parameter, so we use a statistic to estimate it.

A quick way to remember: population goes with parameter, sample goes with statistic.

Standard symbols you will use in this course:

  • population mean μ ("mu"), sample mean x̄ ("x-bar")
  • population standard deviation σ ("sigma"), sample standard deviation s
  • population proportion p, sample proportion p̂ ("p-hat")

Sampling variability and sample size

The curriculum suggests an investigation like this one, using the Census at School database (a collection of anonymous survey answers from students). We imitate it with a computer.

Imagine a school of 500 students in which exactly 150 walk to school, so the population proportion is p = 150/500 = 0.30. A computer drew 20 random samples of 10 students and 20 random samples of 50 students, and found the sample proportion p̂ who walk in each one.

0.00.10.20.30.40.50.60.7n = 100.00.10.20.30.40.50.60.7n = 50true population proportion 0.30sample proportion who walk to school
Each dot is one random sample. With samples of 10, the sample proportions ranged from 0.00 to 0.50. With samples of 50, they ranged only from 0.22 to 0.42, clustered more tightly around 0.30.

What the simulation shows

  • Every sample gave an estimate, but few were exactly right. That is sampling variability, and it is normal. It is not a mistake.
  • The estimates centre on the truth. Random samples do not lean consistently high or low; they scatter around 0.30.
  • Bigger samples vary less. With n = 10 the estimates spread from 0.00 to 0.50. With n = 50 they stayed between 0.22 and 0.42.

The same thing happens with a sample mean or median: larger random samples give statistics that are closer, on average, to the population value. The spread shrinks in proportion to 1/√n, so to cut the spread in half you need about four times as many individuals. You will put a number on this spread with the margin of error in One-Variable Data Analysis.

Random variability versus bias

Sampling variability is random: estimates scatter on both sides of the truth, and a bigger sample reduces it. Bias is different: it pushes results consistently in one direction, and a bigger sample does not fix it. You will study bias in the lesson Bias in Sampling and Surveys.

Worked example

Example 1: Naming the sources of variability

A Grade 12 class grows 20 radish seeds in identical cups on one windowsill. After two weeks, three different students measure the plants with rulers and record heights from 4.1 cm to 9.6 cm. Name each source of variability.

  1. Natural: the seeds are genetically different, so they grow differently even in the same conditions.
  2. Measurement: three students measuring with different rulers may start from slightly different points or read the scale differently.
  3. Induced (uncontrolled factors): cups nearer the middle of the window may get more light; some cups may dry out faster.
  4. Sampling: these 20 seeds are a sample of all seeds of that variety. Another 20 seeds would give a somewhat different mean height.

All four kinds can be present. A good plan (one person measuring, cups rotated daily) reduces measurement and induced variability; natural and sampling variability always remain.

Worked example

Example 2: Comparing sample means of two sizes

Five random samples of 5 adults had mean heights (cm) of 162, 175, 168, 181 and 159. Five random samples of 25 adults from the same population had means of 168, 171, 169, 173 and 170. Compare the variability of the two sets of means.

  1. Range of the n = 5 means: 181 − 159 = 22 cm.
  2. Range of the n = 25 means: 173 − 168 = 5 cm.
  3. The larger samples give means that agree with each other much more closely.
  4. Average of the n = 25 means: (168 + 171 + 169 + 173 + 170) ÷ 5 = 851 ÷ 5 = 170.2 cm, a sensible estimate of μ.

Means from samples of 25 vary far less (range 5 cm) than means from samples of 5 (range 22 cm). Larger samples give more consistent estimates.

Worked example

Example 3: Is the difference real?

A random sample of 10 students from School A has a mean of 7.4 hours of sleep per school night. A random sample of 10 students from School B has a mean of 7.1 hours. A student concludes, "School A students sleep more." Is this justified?

  1. The difference is 7.4 − 7.1 = 0.3 hours, about 18 minutes.
  2. With only 10 students per sample, sampling variability is large. Two samples from the same school could easily differ by 0.3 hours.
  3. To decide whether the difference is real, you would need larger samples and a measure of spread (such as the standard deviation), which you will learn in One-Variable Data Analysis.

Not justified yet. A 0.3-hour difference between two samples of 10 could easily be due to sampling variability alone.

Just for fun

Statistics joke: a statistician stood with one foot in a bucket of ice water and the other in a pot of hot water. "On average," she said, "I feel fine." Which is why we always report variability, not just the average.

Hands-on activity

Card-deck sampling

You need: one standard deck of 52 playing cards (jokers removed), paper and a pencil.

The population is the deck. Exactly half the cards are red, so the population proportion is p = 0.5.

  1. Shuffle well. Deal 4 cards, record the proportion that are red (0, 0.25, 0.5, 0.75 or 1), and return them. Repeat until you have 10 samples of size 4.
  2. Now deal 16 cards at a time, record the proportion red (a number of sixteenths, e.g. 9/16 = 0.5625), and return them. Collect 10 samples of size 16.
  3. Draw two dot plots on the same scale from 0 to 1, one for each sample size.
  4. Compare: Which set of proportions is more spread out? How many samples landed exactly on 0.5?

What to look for: the size-4 proportions usually spread over most of 0 to 1, while the size-16 proportions usually stay roughly between 0.3 and 0.7. Both sets centre near 0.5. Your exact results will differ from everyone else's; that difference is sampling variability.

Your turn

1

Explain in one sentence why variability is the reason statistics exists.

2

You use the mean goals per game from a hockey team's first 10 games to estimate its mean for the whole season. Is the first-10-games mean a statistic or a parameter?

3

Name the source of variability: a scale gives 1.2 kg, 1.3 kg and 1.2 kg for the same bag of flour.

4

Name the source of variability: two random samples of voters give 44% and 47% support for the same candidate.

5

What symbol represents a sample proportion? A population mean?

6

Would you expect the means of samples of size 100 to vary more or less than the means of samples of size 20? Why?

More practice: warm-up

1

What is variability?

2

Is "the mean height of all Grade 12 students in Ontario" a parameter or a statistic?

3

Give an example of natural variability.

4

What does x̄ represent?

5

True or false: a bigger random sample removes all sampling variability.

More practice: on your own

1

In a school of 800 students, 240 play a school sport. (a) What is the population proportion? (b) A random sample of 40 students includes 14 who play. What is p̂? (c) Is the difference a mistake?

2

Five samples of 20 bags of chips have mean masses (g) 39.8, 40.3, 40.1, 39.6, 40.2. Find the range of the means and their average.

3

A researcher wants the spread of her sample means to be about half as large as it is with samples of 30. About how big should her samples be?

4

Two people time the same 100 m sprint by hand and get 13.4 s and 13.7 s. Name the source of variability and suggest one way to reduce it.

5

Why do random samples "centre on the truth" while their individual results vary?

Challenge

1

A population has p = 0.30. Twenty random samples of 10 give proportions that are all multiples of 0.1. Twenty random samples of 50 give multiples of 0.02. Explain why, and why this matters when you compare dot plots.

2

A news story says, "Average rent rose from $1820 to $1850 this month," based on two random samples of 50 listings. Write a short response explaining what else you would need to know before believing that rents really rose.

3

Explain why a very large sample from a biased source (for example, 50 000 responses to an online poll on a sports website about a sports-funding question) can still give a poor estimate, even though its sampling variability is tiny.

Check your understanding

Quiz

1.

Which best describes sampling variability?

2.

Which is a parameter?

3.

What happens to the spread of sample means as the sample size increases?

4.

Two students measure the same table and get 152.3 cm and 152.8 cm. This is an example of:

5.

To make the spread of sample means about half as large, you should multiply the sample size by about:

6.

Which symbol is a sample proportion?

For parents and teachers

Big idea (C1.2): data always vary. Some variability is real (people differ), some comes from measuring, and some comes from which individuals happened to be sampled. Larger random samples give more consistent estimates.

How to help: Do the card-deck activity together; it takes 15 minutes and makes sampling variability concrete. Ask: "If we did this again, would we get the same numbers?" (No, and that is fine.)

Common difficulties:

  • Thinking a sample result that differs from the population value means someone made a mistake.
  • Mixing up parameter (population) and statistic (sample). The p/p and s/s memory trick helps.
  • Believing a larger sample fixes everything. It reduces random variability but not bias; that idea returns in the lessons on sampling and bias.
  • Expecting the spread to halve when the sample size doubles. It takes about four times the sample size.