The missing planes mattered most

If you reinforce only the bullet holes on returning planes, you may protect the wrong places.

The planes that made it home are the survivors. The fatal damage is more likely where returning planes had fewer holes.

By the Math Says Yes editorial team

Human-reviewed under our source and correction standards.

How we review content

THE TRAP

Visible damage grabs attention, but invisible missing cases can carry the real signal.

A plane silhouette covered in bullet-hole dots on the wings and tail, with the engines left unmarked.

What this shows

The returning planes are real data, but they are not the whole dataset — and learning only from the cases that made it into view is . They show where damage was survivable, because planes hit in those places still came home. The missing planes are the counterexample you cannot inspect directly. Good analysis asks what had to happen for a case to appear in the data at all.

Why intuition fails

Visible evidence feels more concrete than missing evidence. A map of bullet holes, a list of successful founders, or a collection of popular products gives the mind something to inspect. The absent cases have no rows, no photos, and no quotes, so they feel less real. flips that instinct: the missing cases may be exactly where the decision-relevant information lives.

Worked example

Returning planes showed bullet holes across the wings and fuselage, so the obvious plan was to armor those damaged areas. Wald's logic reversed the view: the holes marked places where a plane could be hit and still come home, while the planes with fatal engine or cockpit hits were missing from the because they did not return. The quiet, unmarked areas on the survivors were the real candidates for armor. Reinforcing the most damaged visible areas would have spent protection on damage the planes had already survived.

How to use it

When a lesson comes from winners, survivors, reviews, portfolios, or visible examples, first ask what selection rule created the . Who failed before they could be observed? Who had no reason or ability to report back? What outcome would remove a case from the dataset? Those questions turn from a slogan into a practical checklist.

What people get wrong

Successful founders, surviving companies, published books, admitted students, and returned aircraft all passed a filter before you ever saw them. Studying them in rich detail feels rigorous, but detail is not the same as balance: returning planes can be measured carefully and still be a biased . Surveying survivors can reveal what they have in common; it cannot by itself reveal what caused survival, because the failed attempts that shared those same traits are missing from view. A vivid survivor can teach something — it just cannot tell you how many similar attempts disappeared.

When it applies

matters whenever failure removes cases before you inspect the data: business advice based only on winners, investing histories that drop closed funds, product reviews from remaining users, or health stories from people who recovered. Ask what had to happen for a case to be visible, and whether the invisible cases would change the conclusion.

Source note

Wald's aircraft vulnerability work supplies the classic operational example: infer danger partly from the damage patterns on planes that did not come back. The page uses that example as a general selection- lesson.

Try It

Armor the plane

Every dot is a bullet hole reported by a bomber that made it home. Armor is heavy — you can protect two zones. (Illustrative pattern, not Wald's raw data.)

Tap the zones YOU would armor.

holes on survivors

on engines & cockpit

The returning planes are covered in holes — except the engines and cockpit. Choose up to two zones to armor.

FAQ

What is survivorship bias?

Survivorship bias is the error of drawing conclusions from the cases that remain visible while ignoring cases that disappeared before they could be counted. The visible cases may be real and detailed, but they can point to the wrong decision when the missing cases differ in an important way.

Why reinforce the areas with fewer bullet holes?

Because returning planes show damage they survived. Areas with fewer holes on returning planes may be places where hits prevented a plane from returning at all. The absence of holes can be the warning signal.

Where does this show up outside planes?

It shows up in business advice from successful founders, investment portfolios that omit failed funds, product reviews written by unusually motivated users, and any analysis that studies only winners or survivors. The first question is always who did not make it into the dataset.