A reader's field guide to the statistics that pass for facts. Eight ways numbers mislead — each with a small interactive you can actually poke at. Learn to spot them, and you read the news more honestly.
They are true numbers framed to point the wrong way — a real count, a real percentage, a real chart, arranged so the obvious reading is the wrong one. You don't need statistics to defend yourself; you need a handful of questions. Here are the eight that catch most of it. All figures below are illustrative, chosen to make the mechanism visible.
"State A had 5,000 road deaths last year; State B had 1,200." So State A is more dangerous? Only if the two states are the same size. A count means nothing without the population it came from. Switch the view.
Ask: out of how many? A count tells you size; a rate tells you risk. The news loves counts because they sound bigger.
A drug that takes your risk from 2% to 4% has doubled it (+100%, relative) and raised it by 2 percentage points (absolute). Both are true. One sells papers. Enter any two rates and see both framings.
Ask: percent of what base? A scary relative rise on a tiny base is a tiny absolute change.
A test for a rare condition is 99% accurate. You test positive. Your chance of actually having it can still be under 1-in-2 — because when the condition is rare, false positives outnumber true ones. Drag the prevalence and watch.
Ask: how common is it to begin with? Accuracy without the base rate is half the story — this is Bayes' theorem in everyday clothes.
The same four numbers — 48, 49, 50, 52 — can look like a dramatic surge or a near-flat line, depending only on where the y-axis starts. Toggle the baseline.
Ask: where does the axis start? A bar chart should almost always start at zero. When it doesn't, someone is amplifying a small difference.
Give me the freedom to choose the starting year and I can make this wobbly series look like steady progress or steady decline — without changing a single data point. Slide the start year and watch the line that's kept fixed.
Ask: why that start year? An honest trend shows the whole series, not the slice that flatters the argument.
A poll says 51% to 49%. With a small sample, the margin of error swamps the gap — the "lead" is noise. Grow the sample and watch the uncertainty bands shrink until they finally separate.
Ask: how many people, and what's the margin of error? If the gap is smaller than the margin, there is no gap.
"Children with more books at home score higher — so buy books!" Maybe. But richer, more-educated households have both more books and higher scores. Compare all children and the slope is steep. Then compare only children within the same income group — and watch it melt.
Ask: what else could explain both? A genuine cause survives the question "what's the confounder?" — see ImpactMojo's work on causal inference.
In WWII, analysts mapped the bullet holes on returning bombers and proposed armouring the spots that were hit most. The statistician Abraham Wald saw the trap: those planes survived. Armour belongs where the holes aren't — the engines and cockpit — because planes hit there never came back to be counted. Toggle what the data shows.
Ask: who's missing from this data? "Most successful founders dropped out" forgets the far larger crowd of dropouts who failed. Selection decides the story before the numbers do.
