A 95% confidence interval is not a 95% promise

A 95% confidence interval means the method works 95% of the time in repeated samples, not that this exact range now has a 95% chance of holding the truth.

Confidence intervals are honest ranges around estimates, but the confidence level belongs to the long-run method that created the range.

By the Math Says Yes editorial team

Human-reviewed under our source and correction standards.

How we review content

THE TRAP

Treating confidence as a personal probability attached to one finished interval, instead of a reliability rate for the interval-building method.

Several interval bars around a target line, with most bars crossing the target and one missing it.

The range is an estimate

A point estimate says one number: the , the survey share, or the measured difference. A wraps that number in a range to show sampling uncertainty. The range says that the data are compatible with several nearby values, not only the point in the middle. That is why intervals are usually more informative than single estimates: they show both the best guess and how precise the guess is.

What the Numbers Show

intervals that miss the true value (about 5 of 100)

intervals that capture it (about 95 of 100)

Across 100 repeated samples, a 95% interval method captures the true value about 95 times and misses about 5.

Where 95% lives

The 95% refers to the long-run behavior of the procedure. Imagine repeatedly drawing new samples from the same and computing a in the same way each time. Some intervals would miss the true value and most would cover it. A 95% procedure is designed so that about 95 out of 100 such intervals cover the truth in the long run.

The common misread

After one interval is computed, the true value is fixed: it is either inside that range or outside it. The interval does not contain 95% of the data, and it is not automatically a 95% statement about the parameter. That distinction sounds technical, but it matters because overconfident wording can turn uncertainty into a promise the data cannot make.

How to use it

Use confidence intervals to compare precision and plausible values. Wide intervals the estimate is still loose; narrow intervals mean the method has pinned the value down more tightly. If the range includes values that would change your decision, the result is not settled enough for that decision. The interval is not a guarantee, but it is a disciplined way to keep uncertainty visible.

Worked example

Suppose a survey estimates support at 52% with a 95% from 48% to 56%. The interval is not saying there is a 95% that this fixed value is inside this particular range. It is saying that the method used to build ranges like this is designed to capture the true value about 95% of the time across repeated comparable samples.

When it applies

Confidence intervals are strongest when the sampling process and model assumptions are close to the situation that produced the data. They are weaker when selection , nonresponse, dependence, or bad measurement dominates the uncertainty. Do not use a as a range for individual future outcomes; that is a prediction-interval question. Do not read it as a Bayesian credible interval unless the analysis was actually Bayesian.

What people get wrong

The common shortcut is to say "there is a 95% the true value is in this interval." That may sound harmless, but it changes the meaning. In the frequentist setup, the parameter is fixed and the interval is produced by a procedure with long-run coverage.

Source note

Morey, Hoekstra, Rouder, Lee, and Wagenmakers document common confidence-interval fallacies, including treating an interval as a direct statement about a fixed parameter or as a guarantee about future replication. That paper supports the page's warning that intervals are useful uncertainty tools, but only when interpreted through the procedure that created them.

FAQ

Does a 95% confidence interval contain 95% of the observations?

No. It is a range for a population value such as a mean or proportion. A prediction interval is the kind of range meant to describe future individual observations.

Why are confidence intervals better than point estimates alone?

They show precision. Two studies can have the same point estimate but very different ranges, and the wider range should make you less confident about the exact value.