D. Statistical Analysis · Two-Variable Data Analysis and ValidityOutliers and Influential Points
D2.4 D2.5About 65 minutes
Learning goals- I can use residuals to identify outliers in two-variable data.
- I can tell the difference between an outlier and an influential point.
- I can measure the effect of a point by fitting the model with and without it.
- I can decide, and justify, how to handle an unusual point honestly.
Words to know
outlier (in regression)
A point with a large residual: it lies far above or below the pattern of the other points.
influential point
A point whose removal noticeably changes the slope, intercept or r of the model.
leverage
The potential of a point to pull the line toward itself. Points far from x̄ in the x-direction have high leverage.
2s_e rule of thumb
A residual more than about twice the standard error s_e in size is worth investigating as a possible outlier.
Two strange trees
A forestry student measures the age (from ring counts on cores) and trunk diameter of 8 sugar maples in a woodlot. The data follow a strong linear trend. Then she adds two more trees:
- Tree A is 30 years old but only 9 cm across. It grew in deep shade under much taller trees.
- Tree B is 110 years old and 60 cm across, older than any other tree she measured.
Which of these trees causes more trouble for her model? The answer surprises most people.
| Age (years) | 12 | 18 | 25 | 30 | 36 | 42 | 48 | 55 |
|---|
| Diameter (cm) | 11 | 14 | 23 | 24 | 31 | 33 | 41 | 44 |
The original model
For the 8 trees, the least-squares line is ŷ ≈ 0.794x + 1.23 with r ≈ 0.993: a very strong positive linear relationship. Each extra year of age goes with about 0.8 cm more diameter. Older trees have had more years to grow, so this is a genuine cause-and-effect relationship, one of the examples named in the Ontario curriculum.
Outliers: big residuals
In two-variable data, an outlier is a point far from the pattern, which shows up as a large residual. A useful rule of thumb: investigate any point whose residual is more than about 2s_e in size (s_e is the standard error from the last lesson).
Notice the difference from one-variable outliers. A point can have ordinary x and y values on their own and still be an outlier because it does not fit the relationship. Tree A's age (30 years) and diameter (9 cm) are both inside the ranges of the data. It is only the combination that is unusual.
Worked exampleExample 1: Is tree A an outlier?
With tree A included, the line is ŷ ≈ 0.824x − 1.54 and s_e ≈ 5.88. Is A an outlier by the 2s_e rule?
- Predicted diameter for A: ŷ = 0.824(30) − 1.54 ≈ 23.18 cm.
- Residual: e = 9 − 23.18 ≈ −14.18 cm.
- 2s_e ≈ 2 × 5.88 = 11.76.
- |−14.18| > 11.76, and every other residual is under 4 cm in size.
Yes. Tree A's residual (about −14.2 cm) is far beyond 2s_e (about 11.8 cm). It is an outlier. Its effect: r drops from 0.993 to 0.901, but the slope hardly changes, because A is near the middle of the x-values.
On the residual plot, A stands out far below every other point and below the −2s_e line. Notice also that A drags the line down, so most other residuals become positive.Influential points: far out in x
Tree B is different. Its x-value (110 years) is far from the other ages (12 to 55). A point like this has high leverage: like a long lever, it can pull the line toward itself. A point is influential if removing it changes the model noticeably.
Worked exampleExample 2: Is tree B influential?
Compare the models with and without tree B.
- Without B: ŷ ≈ 0.794x + 1.23, r ≈ 0.993.
- With B: ŷ ≈ 0.506x + 10.07, r ≈ 0.952.
- The slope drops by about 36% (from 0.794 to 0.506), and the intercept changes from about 1.2 to 10.1.
- The original model predicts ŷ = 0.794(110) + 1.23 ≈ 88.6 cm for B, so B is about 29 cm below that line. But with B included, B's own residual is only about −5.8 cm.
Yes, B is very influential: one tree changed the slope by more than a third. Influential points can hide in a residual plot because they pull the line toward themselves. The best check is to fit the model with and without the point.
What should you do with an unusual point?
- Check for errors first. Was a number mistyped (91 instead of 19)? Was a measurement made wrongly? Fix real errors.
- Ask whether it belongs to the population you are studying. Tree A grew in deep shade; if your question is about maples in open woodlots, it may not belong. Tree B is much older than the others; perhaps old trees need a separate model (growth often slows as trees age).
- Never delete a point just because it spoils your r. That is a form of data manipulation.
- Be transparent. Report the model with and without the point, and explain your choice.
- Do not extrapolate a model built from young trees to very old trees (next lesson).
Worked exampleExample 3: Writing it up
Write two or three sentences a student could put in a report about trees A and B.
- Name each point, say how it is unusual, and give its effect on the model with numbers.
- Give a reason connected to the context.
- State the decision and how it affects conclusions.
"Tree A (30 years, 9 cm) has a residual of about −14 cm, more than twice the standard error, so it is an outlier; it grew in deep shade, so I excluded it from the model for open-grown trees but report it separately. Tree B (110 years) is influential: including it lowers the slope from 0.79 to 0.51 cm per year. Because it is far older than the other trees and growth may slow with age, my model applies only to trees aged about 10 to 55 years."
Outlier or influential? A quick guide
- Outlier: far from the pattern vertically (large residual). Often reduces |r| a lot.
- High-leverage point: far from the other points horizontally (x far from x̄).
- Influential point: changes the model a lot when removed. High-leverage points that do not follow the trend are usually influential.
- A high-leverage point that lies on the trend is not very influential; it may even raise r.
Just for funStatisticians have a saying: "An outlier is either a mistake or a discovery." Some of the most important findings in science began as a data point that refused to sit on the line.
Hands-on activityMove one point and watch the line
You need: Desmos (free online) or a spreadsheet, or graph paper and a calculator.
- Enter the 8 tree data points and fit the least-squares line. Write down a, b and r.
- Add a 9th point at (30, y) and try y = 25, 15, 9 and 0. Record a, b and r each time.
- Now add a 9th point at (x, 60) and try x = 60, 80, 110 and 150 (remove the previous extra point first). Record a, b and r each time.
- In Desmos you can make the 9th point draggable by typing its coordinates as sliders, such as (p, q), and adding it to the table.
- Write a short paragraph: which kind of point changed the slope more? Which changed r more?
What to look for: a point at x = 30 (near the middle of the data) mainly changes b and r, not the slope. A point far out in x changes the slope a lot, especially when its y-value does not follow the trend.
Your turn
1A model has s_e = 3. Which residuals would you investigate: 2.5, −4.1, 7.2, −6.5?
2s_e = 6. Investigate 7.2 and −6.5 (both larger than 6 in size).
2A point has x and y values that are both close to their means, but its residual is large. Can it be an outlier?
Yes. In two-variable data, an outlier is far from the relationship, not necessarily from the means of x and y.
3Why might an influential point have only a small residual?
It pulls the least-squares line toward itself, which shrinks its own residual.
4Data (1, 2), (2, 4), (3, 6), (4, 8) give ŷ = 2x. A new point (20, 10) is added. Is it likely to be influential? Explain.
Yes. x = 20 is far from the other x-values (high leverage) and far below the trend (the line predicts 40). It will flatten the line a lot.
5A student removes three points from her data to raise r from 0.62 to 0.91 without checking them. What is wrong?
Points should only be removed for a justified reason (an error, or not belonging to the population). Removing them to improve r misrepresents the data.
6Give two legitimate reasons to remove or separate a point.
A recording or measurement error that cannot be fixed; the point is not from the population being studied (for example, a different species or a broken instrument).
More practice: warm-up
1What kind of point is far from the others in the x-direction?
2What is the first thing to check about an outlier?
Whether it is an error (typing or measurement).
4True or false: removing an unusual point always increases r.
False. It depends where the point is. In the lesson on interpreting r, the faraway point (10, 10) raised r from 0 to 0.9, so removing it would lower r.
5What is the best way to measure a point's influence?
Fit the model with and without it and compare.
More practice: on your own
1Points (1, 3), (2, 5), (3, 7), (4, 9) lie on y = 2x + 1. A fifth point (10, 21) is added. Is it influential? Explain.
No (not much): it lies exactly on the line y = 2x + 1 (2(10) + 1 = 21), so the least-squares line stays y = 2x + 1. It has high leverage but follows the trend. (r stays 1.)
2For data with ŷ = 4x + 12 and s_e = 2, a point is (5, 25). Is it an outlier by the 2s_e rule?
ŷ = 32, e = 25 − 32 = −7. |−7| > 4, so yes, investigate it.
3In a scatter plot of house size and price, a mansion is 5 times bigger than any other house. Explain why it could be influential and what a researcher might do.
Its x-value is far from the others (high leverage), so it can tilt the line a lot. The researcher could fit the model with and without it, report both, and perhaps restrict the model to typical house sizes.
4Why is it a problem to quietly delete outliers in a published study?
Readers would think the relationship is stronger and cleaner than it really is. Conclusions could be misleading, and other researchers could not reproduce the results. Any removal should be reported and justified.
5A hospital study of age and recovery time includes one patient aged 99 when everyone else is 20 to 60. With the patient, the slope is 0.5 days per year; without, 0.2. Describe the point and the right reporting.
The 99-year-old is an influential (high-leverage) point: it more than doubles the slope. Report both slopes, explain the change, and limit the main model to ages 20 to 60, where the data are.
Challenge
1Prove that adding a point exactly at (x̄, ȳ) does not change the least-squares slope.
The slope is a = Σ(x − x̄)(y − ȳ) ÷ Σ(x − x̄)². A new point at (x̄, ȳ) adds (0)(0) = 0 to the top and 0² = 0 to the bottom, and it does not change the means. So a is unchanged (and since the line still passes through (x̄, ȳ), b is unchanged too).
2For the 8 trees, the line is ŷ ≈ 0.794x + 1.23. Use it to show that tree B (110, 60) would have a residual of about −28.6 cm relative to the original line, and explain why the residual is so much smaller once B is included in the fit.
ŷ = 0.794(110) + 1.23 ≈ 88.6, so e ≈ 60 − 88.6 = −28.6 cm. Including B lets the least-squares line tilt toward it (slope 0.51), because a point far from x̄ contributes heavily to Σ(x − x̄)(y − ȳ) and Σ(x − x̄)². Its residual shrinks to about −5.8 cm.
3A data set has r = 0.05. Adding one point raises r to 0.80. Describe where that point must be and why this is a warning sign.
The point must be far from the rest in both directions, along a diagonal from the cloud (far up-right or far down-left), so its deviation product is huge. A single point creating almost all the correlation means r describes that one point, not the relationship among the others, as in the lesson on interpreting r.
Quiz
1. In regression, an outlier is a point with:
2. A point far from the others in the x-direction that does not follow the trend is most likely:
3. s_e = 4. Which residual should you investigate?
4. What is the best way to see how much one point affects a model?
5. A point lies far to the right but exactly on the trend line of the other points. Removing it will:
6. Which is an acceptable reason to remove a data point?
Big idea (D2.4, D2.5): unusual points come in two kinds. Outliers sit far from the pattern (big residuals). Influential points, usually far out in x, can pull the line toward themselves and change the slope a lot, sometimes while hiding with a small residual. Always fit with and without, and report honestly.
How to help: Ask your teen to show you the Desmos activity with a draggable point. Watching the line swing as the point moves makes the idea of leverage memorable. Ask, "Would you delete this point? Why?"
Common difficulties:
- Thinking an outlier must have an extreme x or y value on its own.
- Assuming a small residual means a point is harmless.
- Deleting points to improve r without justification.
- Forgetting to report the effect of a point in numbers.