C. Organization of Data for Analysis · Organization of Data for Analysis

The Role of Data in Statistical Studies

C1.1About 65 minutes

Learning goals

I can explain what a statistical study is and describe the five steps of the statistical inquiry process.
I can give examples of how data are used in medical research, political decision making, market research and data mining.
I can explain several reasons why two studies of the same relationship can reach different conclusions.
I can tell the difference between what the data show and a conclusion that goes beyond the data.

Words to know

data
Recorded facts or values, such as numbers, measurements, categories or answers to questions. The singular is datum, though most people now say "data" for both.

statistics
The science of collecting, organizing, analysing and interpreting data to answer questions and make decisions when there is uncertainty. A statistic is also a single number calculated from data, such as a mean.

statistical study
An investigation that uses data from a group of individuals to answer a question about a larger group or a relationship.

statistical inquiry process
The cycle of steps in a study: pose a problem, plan, collect data, analyse, and conclude (sometimes shortened to PPDAC).

variable
A characteristic that can change from one individual to another, such as height, favourite sport or hours of sleep.

data mining
Using computers to search very large data sets for patterns and relationships that nobody asked about in advance.

evidence
Information from data that supports or weakens a claim. Evidence can be strong or weak depending on how the data were collected.

Data is everywhere

This morning, a weather app used data from thousands of stations to tell you whether to wear a toque. A streaming service used your listening history to suggest a song. A grocery store's loyalty card recorded what your family bought, and the store will use that record to decide what to put on sale next week.

Each of these is a small statistical study: someone used data to answer a question and make a decision. In this unit you will learn how good data are collected and organized. In the sibling units One-Variable Data Analysis and Two-Variable Data Analysis and Validity, you will analyse that data. In the Culminating Data Management Investigation, you will run a full study of your own.

Remember from Grade 9: in Mathematics (MTH1W) you looked at large data sets, privacy and data sovereignty, and you planned and carried out a small modelling project. Grade 12 goes deeper: you will learn why a study's design decides how much you can trust its conclusions.

What is statistics?

Statistics is the science of collecting, organizing, analysing and interpreting data. Its job is to help people make good decisions when they cannot be certain.

Why is there uncertainty? Because we almost never have data for everyone or every situation. A drug company tests a medicine on a few thousand patients, but millions may use it. A polling company calls about 1000 voters, but millions will vote. Statistics tells us how far we can trust a conclusion that reaches beyond the data we actually have.

A statistical study follows a cycle of five steps. Some textbooks call it PPDAC: Problem, Plan, Data, Analysis, Conclusion.

The statistical inquiry process. A good conclusion usually raises new questions, so the cycle starts again.

The five steps in detail

Problem. Pose a clear question that data can answer. "Are teenagers on their phones too much?" is vague. "How many hours per school day do Ontario Grade 12 students use their phones?" is a statistical question.
Plan. Decide what to measure (the variables), who to study (the population and sample), and how to collect the data (survey, experiment, existing database).
Data. Collect the data carefully and organize it in tables, spreadsheets or databases. Check for errors.
Analysis. Make graphs and calculate statistics such as means, medians and correlation.
Conclusion. Answer the question in context. Say how sure you are, what the limitations were, and what you would study next.

The plan is the step people most often rush. A poor plan cannot be rescued by clever analysis: as statisticians like to say, garbage in, garbage out.

Where data make a difference

The curriculum names several fields where statistical studies are essential:

Medical research. Clinical trials compare a new treatment with an existing treatment or a placebo. Health agencies use the results to decide whether a drug is approved.
Political decision making. Governments use census counts and survey data to plan schools, hospitals and transit, and to decide how money is shared. Federal electoral boundaries are reviewed using census population counts.
Market research. Companies survey customers and test products to decide what to sell, how to price it and how to advertise it.
Data mining. Computers search huge data sets (loyalty-card purchases, streaming histories, bank transactions) for patterns. A bank might discover that a sudden purchase in another country is a sign of fraud.

Other fields include sports analytics, insurance, environmental science (snowfall records, lake-ice dates), education and engineering.

Field	Question	Data used	Decision
Medical research	Does a new asthma inhaler work better than the standard one?	Lung-function measurements from patients randomly given one inhaler or the other	Approve or reject the new inhaler
Political decision making	Where will the town need a new elementary school in 10 years?	Census counts of young children by neighbourhood; housing plans	Where to build
Market research	Which of three new granola-bar flavours will sell best?	Taste-test ratings and sales in trial stores	Which flavour to launch across Canada
Data mining	Which customers are likely to cancel their phone plan?	Millions of account records (usage, complaints, plan type)	Who gets a special offer

Four kinds of statistical study. Each one moves from a question, through data, to a decision.

Why can studies of the same relationship disagree?

You will often read two news stories that seem to contradict each other: "Coffee is good for your heart" and "Coffee is bad for your heart." The curriculum's example is studies of whether increasing jail sentences reduces crime rates. Researchers have reached different conclusions. That does not mean statistics is useless. It means the studies were different in ways that matter:

Population: different provinces, countries, age groups or time periods.
Definitions and measurement: "crime" could mean crimes reported to police, or crimes people describe in victimization surveys (many crimes are never reported).
Study design: an observational study only watches what happens; an experiment deliberately changes something. Observational studies cannot easily separate cause from coincidence.
Other variables: crime also depends on the economy, policing, age of the population and more. A study that ignores these can credit (or blame) the wrong thing.
Sample size and sampling method: small or non-random samples give less reliable results.
Analysis and reporting choices: which statistics were calculated, and which results were published.
Bias and funding: who paid for the study and what they hoped to find.

A hypothetical illustration: two studies ask the same question but use different populations, time spans, measures and methods, so they can reach different conclusions.

Worked example

Example 1: Two homework studies disagree

Study A surveyed 40 students at one Ottawa high school and found that students who did more homework had higher marks. Study B used data on 5000 students from across Canada and found only a weak relationship between homework time and marks. Give four reasons the studies might disagree.

Sample size: 40 students is a small sample, so chance alone can create or hide a pattern. 5000 students gives a much steadier result.
Population: Study A describes one school. That school's homework policy, courses or teachers may be unusual. Study B covers many kinds of schools.
Measurement: each study may define "homework time" differently (self-reported minutes? assigned work?) and may use different marks (one course, or an overall average).
Other variables: students who do more homework may also have more support at home, fewer part-time job hours or more interest in school. Those variables, not homework itself, could explain some of the pattern. The two studies may have accounted for these differently.
Design: both are observational. Neither assigned homework at random, so neither can prove that homework causes higher marks.

The disagreement is explained by differences in sample size, population, measurement, other variables and design. Study B is broader, but neither study alone proves cause and effect.

Worked example

Example 2: Spotting the five steps

A town council wants to know whether residents would use a new outdoor skating rink. Staff mail a short survey to 600 randomly chosen households, enter the 410 replies into a spreadsheet, find that 62% of replying households would use the rink at least twice a month, and recommend building it while noting that 190 households did not reply. Label each part with a step of the inquiry process.

Problem: Would residents use a new outdoor rink?
Plan: a mailed survey to 600 randomly chosen households.
Data: the 410 replies entered into a spreadsheet.
Analysis: calculating that 62% of replying households would use it at least twice a month.
Conclusion: recommend building it, with a stated limitation (190 households did not reply, so the result might not represent everyone).

All five steps appear. Notice that a good conclusion includes its limitations.

Worked example

Example 3: Does the conclusion go beyond the data?

A headline says: "Study finds students who eat breakfast earn higher marks, so eating breakfast raises marks." The study compared the marks of students who said they ate breakfast with those who said they did not. Is the conclusion justified?

What the data show: in this group, breakfast eaters had higher average marks. That is a relationship (an association).
What the headline claims: breakfast causes higher marks. That is a stronger claim.
The study only observed students; nobody was assigned to eat breakfast. Families who make time for breakfast may differ in other ways (sleep schedules, routines, time to help with homework). These other variables could explain the difference.
To test cause, you would need an experiment, or careful methods that account for other variables (you will study this in Two-Variable Data Analysis and Validity).

No. The data show an association, not cause and effect. A fair headline would be: "Students who eat breakfast tend to have higher marks."

Questions to ask about any study

Who was studied, and how were they chosen?
What exactly was measured, and how?
How many individuals were studied?
Was it an experiment or an observational study?
Who paid for or ran the study?
Does the conclusion match what the data can show?

Just for fun

A statistician walks into a coffee shop that claims "9 out of 10 customers recommend our muffins." She asks, "What did the tenth customer say, and how many customers did you ask?" The barista says, "Ten." She orders tea.

Hands-on activity

Data detective for a day

You need: a notebook or phone, and about 20 minutes spread over one day.

Over one day, notice five places where data are being collected or used around you. Ideas: a weather forecast, a hydro or water bill that compares this month with last year, a nutrition label, a grocery receipt or loyalty app, a fitness tracker, a streaming recommendation, a school report card.
For each one, write: the question someone was trying to answer, the data they used, and the decision that the data helped someone make.
Choose one and rewrite its question as a clear statistical question (it should name a group, a variable and something you could measure).
Ask a family member whether they trust that data source, and why or why not. Write down two things that would make it more or less trustworthy.

What to look for: a strong entry names a specific variable ("kilowatt-hours used per month", not just "electricity"). A strong statistical question looks like "What is the typical number of kilowatt-hours our household uses per month in winter compared with summer?"

Your turn

Put these steps of a study in order: analyse the data; pose a question; draw a conclusion; collect the data; make a plan.

Is "Do students like music?" a good statistical question? If not, improve it.

Give one example of how data are used in medical research.

What is data mining? Give an example.

Two studies of whether a new bike lane reduced collisions disagree. Study A looked at one year before and after; Study B looked at five years before and after. Why might they disagree?

A survey of 1000 Canadian adults found that 48% support a policy. Why is this conclusion uncertain?

State the difference between an observational study and an experiment.

More practice: warm-up

What does the P stand for in each step of PPDAC that begins with P?

Name the step in which you make graphs and calculate means.

True or false: a conclusion should include its limitations.

Give one field, other than medicine, that relies on statistical studies.

What is a variable?

More practice: on your own

A cereal company asks shoppers in one mall to taste a new cereal; 85% like it. The company concludes that 85% of Canadians will like it. Give two reasons this conclusion may be wrong.

A city wants to decide where to add bus routes. Describe a possible plan (variables and data source).

A study of 30 people finds that people who own a dog sleep worse. A study of 20 000 people finds no difference. Which result would you trust more, and why?

Explain the phrase "garbage in, garbage out" in statistics.

Researchers in two provinces study whether raising the minimum wage changed teen employment and reach different conclusions. Give three possible reasons.

Challenge

A loyalty-card data-mining program finds that customers who buy maple syrup also tend to buy pancake mix. The store puts the two side by side and syrup sales rise. Does this prove the placement caused the rise? What else could explain it?

Write a one-paragraph plan for a study answering: "Do Grade 12 students who work part-time sleep less on school nights?" Name the variables, the data source and one limitation.

Two studies of the same new drug disagree: one, paid for by the drug's maker, finds a large benefit; an independent study finds a small benefit. List what you would check before deciding which to trust.

Check your understanding

Quiz

Which step of the statistical inquiry process comes right after posing the problem?

Which is the best statistical question?

A bank's computer searches millions of transactions to find patterns that signal fraud. This is an example of:

Two studies of the effect of jail sentences on crime rates reach different conclusions. Which is not a reasonable explanation?

A study observes that people who drink more water have fewer headaches. What can it conclude?

Why should a conclusion include its limitations?

For parents and teachers

Big idea (C1.1): data are used to make decisions in medicine, government, business and daily life, and different studies of the same question can disagree because they differ in population, measurement, design, sample size and other variables.

How to help: When you see a news story that quotes a study ("A new study shows..."), read it together and ask the questions from the tip box: who was studied, how many, how they were chosen, what was measured, and whether the headline goes further than the data. Ask your teen to name the five steps of the inquiry process for that story.

Common difficulties:

Treating any association as proof of cause and effect.
Writing vague questions. A statistical question names a group and a measurable variable that varies.
Thinking that when two studies disagree, one must be "lying". Usually they studied different things in different ways.
Skipping the plan step. Most problems with studies start there.