Lesson 7 of 8 · 15 min
What 95% confidence means
Explain the confidence level through repeated sampling, see why most but not all intervals contain μ, avoid the probability misreading, and use an interval to judge a claim.
What you will be able to do
- Explain that confidence intervals vary from sample to sample and that most, but not all, contain .
- Interpret a confidence level as the long-run proportion of intervals that capture .
- Count intervals that miss , and find probabilities about them with the binomial distribution.
- Use a confidence interval to judge a claim about , worded carefully in context.
A bottling plant's machine is set to fill bottles with a mean of mL. Imagine 100 inspectors, each taking their own random sample of 40 bottles and building a 95% confidence interval for the mean fill. Their intervals will all be different, because their samples are. Will every interval contain ?
Draw 100 intervals and count the ones that miss . Then change the confidence to 80% and 99%: what happens to the widths, and to the number that miss? Then change .
Twenty 95% intervals from twenty samples of 40 bottles. Nineteen contain ; one (red) misses. (Each sample's own makes real widths differ slightly; here they are drawn equal.)
Key idea
What the confidence level describes
is a fixed number. The intervals are what vary, because they are built from random samples. A 95% confidence level means that if you took many random samples and built an interval from each, about 95% of the intervals would contain . Most, but not all: about 1 in 20 misses.
For one calculated interval, say "we are 95% confident that lies between … and …". The confidence is in the method that produced it.
Common mistake
"There is a 95% probability that μ is in (496.1, 499.5)"
Once the interval is calculated, nothing in that statement is random: is fixed and the endpoints are numbers, so either is in the interval or is not. The 95% is the probability, before sampling, that the method will produce an interval containing . Also avoid "95% of the data lie in the interval" and "95% of sample means lie in the interval": both are wrong.
A random sample of 60 apples gives an approximate 95% confidence interval for the mean mass of all apples in an orchard of g. Which statement is correct?
Fifty independent random samples are taken, and an approximate 90% confidence interval for is calculated from each. How many of the intervals would you expect not to contain ?
Confidence intervals are how scientists and regulators test claims. If a claimed value of lies inside the interval, it is a plausible value and the data give no reason to doubt it (they don't prove it, either). If it lies outside, the data are evidence against the claim, because a 95% interval misses the true mean only about 5% of the time.
Worked example
Testing a label claim with a confidence interval
A drinks company labels its bottles "500 mL" and claims the mean fill is mL. A consumer group buys a random sample of bottles and finds mL and mL.
(a) Calculate an approximate 95% confidence interval for the mean fill , correct to 2 decimal places.
(b) Does the interval support the company's claim? Explain.
(c) The company replies that a 99% interval contains 500 mL. Verify this, and evaluate the company's argument.
Step 1: Calculate the 95% interval
Step 2: Compare with the claim
lies above the whole interval. A 95% interval misses the true mean only about 5% of the time, so the sample is evidence that the mean fill is less than 500 mL.
A value outside the interval is not a plausible value of at this confidence level.
Step 3: Calculate the 99% interval
It does contain , just.
Step 4: Evaluate the argument
The company is right that 500 is inside the 99% interval, but only at the very edge, and it is outside the 95% interval. A higher confidence level makes intervals wider, so it can always be raised until a claim fits. The evidence that the mean is below 500 mL is moderate: strong enough to question the claim at 95%, not at 99%. A larger sample would settle it.
Try it yourself first, then check the working one step at a time.
Answer
(a) mL. (b) No: 500 mL is outside the interval, so the data are evidence that the mean fill is less than 500 mL. (c) The 99% interval is , which just contains 500; the claim is plausible only at the higher confidence level, so the evidence against it is moderate rather than conclusive.
Method
Build the interval, then ask whether the claimed is inside.
Inside: plausible. Outside: evidence against the claim.
Remember that the conclusion can depend on the confidence level, and say so.
Watch out for
Saying the claim is "proved false": the interval gives evidence, not certainty.
Concluding the 95% interval has a 95% chance of containing 500 mL.
A company claims its delivery drivers average 42 deliveries a day. A random sample of 45 driver-days gives the 95% interval for the mean. What is the best conclusion?
Summary
Intervals vary from sample to sample; does not.
A 95% confidence level means about 95% of intervals from repeated samples contain : most, but not all.
Say "95% confident", not "95% probability that is in this interval".
The number of -level intervals out of that contain is approximately .
A claimed value inside the interval is plausible; outside, the data are evidence against it.
Finished the lesson?
Try the 3 quick checks above and the lesson completes itself, or mark it complete now.