Drawing Conclusions: Strength of Evidence, Limitations and Next Steps

Sample: "In this sample of Grade 12 students at Pinecrest, those with part-time jobs slept about 1.1 hours less on school nights, on average, than those without jobs."

None of 10 000 shuffles gave a difference of 1.14 h or more, so a difference this large would be very unlikely if job status had no connection with sleep. Chance alone is not a believable explanation.

Her study is observational; lurking variables (for example, family responsibilities or personal habits) might explain both work and sleep. Only a randomized experiment can show cause.

Grade 12 students at Pinecrest (at that time of year), the population her random sample came from.

p̂ = 0.55. ME = 1.96 × √(0.55 × 0.45 ÷ 40) ≈ 1.96 × 0.0787 ≈ 0.154, so about 55% ± 15%.

Sample answers: the 300 plays were simulated, not played by real carnival-goers (real dice could be slightly unfair, or players might bend the rules); 300 plays is enough to check the average but a single evening of fewer plays could still lose tickets.

"Is associated with."

Sample size and bias (and, for games, the number of times the game is played).

No. It is a valid result if the method was sound.

About −0.62.

For example a larger, multi-school stratified sample, a sleep diary study, or a longitudinal study.

"In this sample of 25 students, there was a very weak positive association (r ≈ 0.12) between music practice time and math marks; practice time explains only about 1% of the variation in marks (r² ≈ 0.014). The data give no meaningful evidence of a relationship, and an observational study could not show cause."

About 18% of shuffles matched or beat the observed difference, so chance could easily explain it. The data do not give convincing evidence of a real difference between the groups.

n(1 − p̂) = 18 × 0.111 = 2, less than 5, so the sampling distribution is skewed and the normal approximation does not hold.

If the 5 non-responders (and 3 excluded) were mostly busy workers who sleep little, the job group's true mean would be lower than observed, making the true difference larger. If they were students without jobs who sleep a lot, the difference would also be larger. If non-responders were unlike either pattern, the effect could go the other way; the direction is uncertain, so it is a limitation to state.

With only 30 plays, random variation is large, so one unlucky run can produce a loss. The data are too few to contradict the theory. The student should report the result, compare it with the expected range (for example using a standard deviation), and play or simulate many more times before concluding anything.

1.14 ÷ 0.27 ≈ 4.2 standard deviations. For a roughly normal distribution of chance differences, a value more than 4 SD from the centre is extremely rare (well under 0.01%), which agrees with 0 of 10 000 shuffles: very strong evidence that the difference is not due to chance alone.

Sports are more common in the no-job group, the group that sleeps more, so if anything sports would reduce their sleep and shrink the difference. It cannot explain why the no-job group sleeps more. A variable that could: family responsibilities or financial pressure (students who need to work may also have caregiving duties at home), or chronotype (night owls might seek evening jobs and also sleep less).

"A sample of 40 does make my estimates imprecise: for example, the proportion of working students who meet the guideline is 27% ± 19%. But the difference I found is large relative to the variation (about 1.7 standard deviations), it appears consistently in the means, medians, box plots and proportions, and my shuffle simulation showed that chance alone almost never produces a difference this big. So the sample is large enough to detect this association, though not to estimate it precisely. A larger, multi-school sample would narrow the margins of error."