Chapter 05 · Field guide

How Do Statistics in X Posts Mislead?

Short answer

Most misleading statistics on X are accurate numbers missing their context: a relative change without the baseline, a percent change dressed up as percentage points, or a test result without the base rate.

Key takeaways

  • A drop from 2 in 100 to 1 in 100 is a 50% relative reduction and a 1 percentage point absolute reduction. Both are true; only one sounds dramatic.
  • Going from 4% to 6% is a rise of 2 percentage points and also a 50% increase. Watch which one a post picks.
  • A 99% accurate test for something that affects 1% of people is right only about half the time when it says yes.
  • Before trusting any number, ask: out of how many, compared with what, and over what period?

A post reads: "Eating blueberries every day cuts your risk of a rare eye condition by 50%!" It might be true. It might also describe a change from 2 cases in 10,000 people to 1. Both statements can come from the same study. This chapter walks through the common ways a correct number tells an incorrect story.

What is the difference between relative and absolute risk?

Relative risk tells you how much a risk changed compared with where it started, while absolute risk tells you how many people out of a group are actually affected.

Take a made-up supplement trial. In the group without the supplement, 2 people in 100 caught a cold during the study. In the group with it, 1 person in 100 did.

  • Relative reduction: the risk went from 2 to 1, which is half. So, "50% fewer colds."
  • Absolute reduction: 2% minus 1% is 1 percentage point. So, "1 fewer cold per 100 people."
  • Number needed to treat: 1 divided by 0.01 is 100. A hundred people take the supplement for one of them to avoid a cold.

All three are honest. The headline usually picks the first because it is the biggest. The FDA's evidence-based guide to risk communication recommends giving absolute numbers alongside relative ones for exactly this reason. Plug any "X% lower risk" claim into the percentage check with a baseline and you will see the absolute version in a second.

Why does percent versus percentage points matter?

A percentage point is the plain difference between two percentages, while a percent change compares that difference with the starting value, so the same change can sound small or large.

Say the share of households in a town owning a heat pump rose from 4% to 6%. That is a 2 percentage point rise. It is also a 50% increase, because 2 is half of 4.

Change Percentage points Percent change
4% to 6% +2 points +50%
40% to 42% +2 points +5%
0.1% to 0.3% +0.2 points +200%

Notice the last row. Tiny starting values produce enormous percent changes. When a post says something "tripled," check whether it tripled from almost nothing.

How does ignoring the base rate fool people?

When something is rare, even a very accurate test produces many false alarms, so a positive result can be far less meaningful than the accuracy figure suggests.

Imagine a screening test that is 99% sensitive (it catches 99% of real cases) and 99% specific (it correctly clears 99% of people without the condition). The condition affects 1% of people. Now test 10,000 of them:

  1. 100 people have the condition. The test flags 99 of them.
  2. 9,900 people do not. The test wrongly flags 1% of them, which is 99 people.
  3. In total, 198 people test positive. Only 99 of them have the condition.

So a positive result is right about 50% of the time, not 99%. The same math applies to anything that "detects" rare events, from spam filters to fraud alerts. The base rates tab in the percentage check runs this calculation with your own numbers.

What questions expose a missing denominator?

Ask "out of how many?" every time a post gives a raw count, because a count without a total cannot tell you whether something is common or rare.

"Over 3,000 people reported headaches after using this app's blue light filter." Out of how many users? If the app has 30 million users, that is 1 in 10,000, possibly lower than the background rate of headaches. If it has 5,000 users, that is most of them.

Watch for these related gaps:

  • No time frame. "500 incidents" per day and per decade are very different.
  • No comparison group. A rate means little without the rate in people who did not do the thing.
  • Changing totals. More reported cases can mean more cases, or simply more testing.

When do averages and small samples mislead?

A mean can be dragged far from typical by a few extreme values, and a small sample can show a dramatic result by chance alone.

Picture ten fans in a fantasy football league. Nine won $40 this season and one won $4,000. The mean winnings are $436, yet nobody typical came close. The median, the middle value, is $40. Government statistics such as household income in the U.S. are commonly reported as medians for this reason.

Small samples swing too. If a running app tests a new training plan on 12 people and 9 get faster, that is 75%. Repeat it with another 12 and you might get 5. Look for the sample size, whether anyone repeated the result, and whether a range of uncertainty is shown. Our chapter on reading a cited study covers what to check in the paper itself.

What is a quick routine for any number in a post?

Rewrite the number as "X out of Y people, compared with Z, over this period," and see whether it still sounds as impressive.

  1. Find the baseline. What was the risk or rate before?
  2. Convert to absolute terms, per 100 or per 1,000.
  3. Check whether "percent" really means percentage points.
  4. Ask how rare the thing is before trusting a test or detector.
  5. Look for the sample size and the comparison group.

If the post does not give enough to do this, that is your answer: you cannot judge the claim yet. Charts have their own set of tricks, covered in misleading charts.

Frequently asked questions

Is relative risk a lie?

No. Relative risk is a legitimate measure and researchers use it constantly. It becomes misleading when it is shown without the baseline, because the same 50% can mean one extra case in 100 or one in a million.

What is the number needed to treat?

It is how many people need an intervention for one extra person to benefit, calculated as 1 divided by the absolute risk reduction. If risk falls from 2% to 1%, the absolute reduction is 0.01 and the number needed to treat is 100.

Why do averages mislead?

An average (mean) is pulled up or down by a few extreme values. If nine people earn $40,000 and one earns $4 million, the mean is about $436,000 while the median is $40,000. Ask which one a post is using.

How small is too small for a sample?

There is no single cutoff, but results from a few dozen people swing a lot by chance. Look for the sample size, whether the result was repeated, and how wide the uncertainty range is.

Sources

  1. U.S. FDA, Communicating Risks and Benefits: An Evidence-Based User's Guide
  2. U.S. Census Bureau, Income in the United States (median household income reporting)
  3. Cochrane, Evidence-based health care resources

Examples in this chapter are invented for teaching. Facts about platforms were checked in November 2026 and can change. See how we research.