A trend can reverse when groups are combined

A treatment can look better in every subgroup but worse overall.

Aggregation can hide the different sizes and risk levels of the groups being combined.

By the Math Says Yes editorial team

Human-reviewed under our source and correction standards.

How we review content

THE TRAP

The total feels like it should summarize the parts. Sometimes the mix of the parts changes the answer.

Two grouped sets of bars rising, but a single large arrow over the combined total points down.

What this shows

A treatment can win inside every subgroup — among the mild cases and the severe cases alike — and still lose in the combined table. That reversal is Simpson's paradox, and it is not a math error. It happens because the groups being combined have different sizes, baselines, or risk levels. When those groups are mixed, the larger or riskier group can dominate the total and hide the pattern that appears inside each subgroup.

What the Numbers Show

93%

A · mild

87%

B · mild

73%

A · severe

69%

B · severe

78%

A · overall

83%

B · overall

A wins mild (93–87) and severe cases (73–69), but treats more severe cases; mixed totals flip to B 83–78.

Why intuition fails

The total feels authoritative because it uses all the data. But totals also throw away labels. If one treatment is attached to many easy cases and the other to many hard cases, the aggregate comparison is partly a comparison of difficulty, not just treatment. The hidden variable is not always severity; it could be age, region, risk, timing, or any factor that changes both group membership and outcome.

Worked example

The chart uses illustrative figures: overall, treatment B succeeds more often, 83% versus 78%. Yet treatment A wins inside both severity subgroups. That can happen if B was used more often on easier cases while A was used more often on harder cases. The overall number then mixes with case mix. The subgroup view and the aggregate view are both calculated correctly, but they are answering different comparisons.

How to use it

When a total contradicts the subgroup pattern, pause before choosing a side. Ask what variable split the subgroups and whether that variable matters to the outcome. Then decide which comparison matches the decision you need to make. Sometimes the aggregate is the right answer; sometimes the subgroup comparison is. The point is to make that choice deliberately instead of letting the spreadsheet choose it for you.

What people get wrong

A total built from thousands of cases feels like it must beat any subgroup view — more data, more truth. But more data helps only if the comparison is fair. If the groups have different mixes of easy and hard cases, the aggregate can be precise and still misleading. The question to ask is not just "how many cases?" but "which kinds of cases are being combined?"

When it applies

Watch for Simpson's paradox when groups differ in baseline risk, exposure, difficulty, age, region, timing, or selection. It matters in treatment comparisons, admissions rates, product metrics, sports, and hiring funnels. Subgroups are not automatically more trustworthy than totals; ask which comparison matches the causal or decision question.

Source note

Simpson's 1951 paper is the primary source behind the contingency-table warning. It supports the page's central claim that interaction and grouping can change how a table should be interpreted, especially when a hidden variable affects both group membership and outcome.

Try It

Simpson's Paradox mixer

Shift which cases Treatment A handles.

Severe cases in A

50%

83%

Treatment A overall

78%

Treatment B overall

With 50% of A's cases severe, A leads overall (83% vs 78%) — matching its edge inside both the mild group (93% vs 87%) and the severe group (73% vs 69%). Push more hard cases onto A and watch the totals cross.

FAQ

What is the main statistical idea?

The main idea is that aggregation changes weights. A subgroup result tells you what happened inside comparable slices; the total also reflects how large those slices are. If the slice sizes differ across groups, the total can reverse the subgroup pattern.

Does Simpson's paradox mean totals are useless?

No. Totals can be exactly the right view when the total population is the decision target. Simpson's paradox means you should check whether an important variable is being mixed away before treating the total as the whole story.

What should I check first?

Check whether the groups being combined are comparable. Look for a lurking variable that affects both group membership and the outcome, such as risk level, difficulty, timing, or selection into one treatment rather than another.