Glossary term
Statistical Significance
A label meaning a result would be unusual if pure chance were the only thing going on — typically a p-value below 0.05.
Visual intuition
ordinary under model
surprise zone
observed
A small p-value lands far into the surprise zone
'Significant' just means the observed result lands in the surprise zone — too far out to blame comfortably on chance.
Example
In an illustrative trial, 60 of 100 treated patients recover versus 45 of 100 on placebo — a near 0.03, below the usual 0.05 line, so the difference would be called statistically significant.
How It Works
Flip a coin 100 times and get 61 heads: if the coin were fair, a result that lopsided would be rare, so we call it statistically significant. That is all the label certifies — hard to blame on alone. It does not the effect is large, important, or proven: with a huge a trivial 0.2-point difference can be significant, and with a small sample a big real effect can fail the test.
Concepts
P-Values
A p-value is the probability of seeing data at least this extreme if a specific null model were true. It is not the probability that the claim is false, not the chance the result was random, and not a measure of how large or important the effect is.
Statistical Power
Statistical power is the chance that a study will detect an effect of a given size if that effect is real. Low-powered studies can miss real effects, while very large studies can detect effects too small to matter in practice.
Related Facts
Data Tricks
4 min · medium
A p-value is not the chance your result is wrong
A p-value is a check for surprise: it asks how often a result this extreme would appear if there were really no effect. It does not tell you the chance your result is wrong.
Risk & Decisions
4 min · medium
Statistical power: a small study can miss a real effect
A study can report 'no clear effect' simply because it was too small and noisy to spot a real one.