Math Says Yes

Concept

Sampling Bias

A distorted result caused by looking at data that does not represent the population you care about. More data does not fix the problem if the same kinds of cases are still missing. Before trusting a conclusion, ask who had a chance to appear in the sample and who was left out.

Lessons

A highlighted window showing only part of a larger hidden sample.

The Sample You See Matters

Your conclusion can be wrong when the data you see excludes important missing cases.

Related Facts

4 min · medium

A friendship network where a few hub dots connect to many others.

Why your friends seem more popular

Pool every name occurrence from every friend list and well-connected people appear more often. The resulting average is never below the uniformly random-person average, and is higher when friend counts vary.

4 min · medium

A stack of published positive studies on a desk and muted null studies hidden in a file drawer.

Publication bias hides the studies that found nothing

If only exciting results get published, the visible research record can make an effect look cleaner and stronger than it really is.

A hospital doorway selecting points from two independent risk clouds, creating a tilted pattern inside.

Berkson's paradox: hospitals can create fake correlations

Two unrelated risk factors can look negatively related inside a hospital, because either one can be the reason a patient got selected into the sample.

4 min · medium

Two grouped sets of bars rising, but a single large arrow over the combined total points down.

A trend can reverse when groups are combined

A treatment can look better in every subgroup but worse overall.

A plane silhouette covered in bullet-hole dots on the wings and tail, with the engines left unmarked.

The missing planes mattered most

If you reinforce only the bullet holes on returning planes, you may protect the wrong places.