The mean is the first number anyone computes and the wrong one surprisingly often. This lesson is about when it is fine, when it is not, and what to look at instead.
When the mean is fine, and when it is not
import numpy as np
import pandas as pd
rng = np.random.default_rng(13)
heights = rng.normal(170, 8, 5000) # symmetric
incomes = rng.lognormal(mean=10.2, sigma=0.9, size=5000) # right-skewed
summary = pd.DataFrame({
'heights': [heights.mean(), np.median(heights), heights.std()],
'incomes': [incomes.mean(), np.median(incomes), incomes.std()],
}, index=['mean', 'median', 'std']).round(1)
print(summary)
print()
print(f'heights: mean / median = {heights.mean() / np.median(heights):.3f}')
print(f'incomes: mean / median = {incomes.mean() / np.median(incomes):.3f}')
Open this example in the editor and change it. Nothing you type is saved.
For the heights the mean and median agree to a fraction of a percent. For the incomes the mean is around 50% above the median, because a small number of very large values pull it up.
The rule: the mean describes the data when the distribution is roughly symmetric and has no long tail. Almost nothing in business is. Revenue per customer, session length, time to resolution, page load time, order value - all are right-skewed, and for all of them the mean describes a customer who does not exist.
The sensitivity to a single value is worth seeing:
import numpy as np
salaries = np.array([28, 31, 33, 35, 36, 38, 41, 44, 48, 52]) * 1000
print(f'mean {salaries.mean():>9,.0f}')
print(f'median {np.median(salaries):>9,.0f}')
print()
with_founder = np.append(salaries, 2_000_000)
print('after one founder joins:')
print(f'mean {with_founder.mean():>9,.0f} ({with_founder.mean() / salaries.mean() - 1:+.0%})')
print(f'median {np.median(with_founder):>9,.0f} '
f'({np.median(with_founder) / np.median(salaries) - 1:+.0%})')
Open this example in the editor and change it. Nothing you type is saved.
One value out of eleven moves the mean by nearly 500% and the median by 3%. That is the whole argument, in two numbers.
Percentiles say more than either
import numpy as np
import pandas as pd
rng = np.random.default_rng(17)
# Page load times: mostly fast, with a slow tail. This is what every
# latency distribution looks like.
latency = np.concatenate([
rng.gamma(2.0, 90, 9_500), # the normal case
rng.gamma(2.0, 900, 500), # the tail
])
percentiles = [50, 75, 90, 95, 99, 99.9]
values = np.percentile(latency, percentiles)
table = pd.DataFrame({'percentile': [f'p{p:g}' for p in percentiles],
'ms': values.round(0)})
print(f'mean {latency.mean():.0f}ms')
print(table.to_string(index=False))
print()
print(f'{(latency > latency.mean()).mean():.1%} of requests are slower than the mean')
Open this example in the editor and change it. Nothing you type is saved.
The mean sits somewhere around the 70th percentile. Reporting it as "typical" means a quarter to a third of your users have a worse experience than the number you published.
For anything a user waits for, report p50, p95 and p99. p50 is the typical experience, p95 is the bad day, and p99 is the complaint. A mean of 300ms with a p99 of nine seconds is a system with a serious problem that the mean conceals entirely.
The same reasoning applies to money:
import numpy as np
import pandas as pd
rng = np.random.default_rng(19)
order_value = rng.lognormal(mean=3.6, sigma=1.1, size=20_000)
print(f'mean {order_value.mean():.2f}')
print(f'median {np.median(order_value):.2f}')
print()
shares = pd.DataFrame({
'top 1%': [order_value[order_value >= np.percentile(order_value, 99)].sum()],
'top 10%': [order_value[order_value >= np.percentile(order_value, 90)].sum()],
'all': [order_value.sum()],
})
print(f'top 1% of orders are {shares["top 1%"][0] / shares["all"][0]:.1%} of revenue')
print(f'top 10% of orders are {shares["top 10%"][0] / shares["all"][0]:.1%} of revenue')
Open this example in the editor and change it. Nothing you type is saved.
Look at the distribution first
Summary statistics can be identical for data that looks nothing alike.
import numpy as np
import pandas as pd
rng = np.random.default_rng(23)
one = rng.normal(100, 15, 4000)
two = np.concatenate([rng.normal(72, 6, 2000), rng.normal(128, 6, 2000)])
three = np.concatenate([rng.normal(97, 8, 3900), rng.normal(180, 5, 100)])
frame = pd.DataFrame({'one': one, 'two': two, 'three': three})
print(frame.describe().round(1))
print()
# describe() says they are similar. A histogram says they are not.
for name in frame.columns:
values = frame[name]
counts, edges = np.histogram(values, bins=14, range=(50, 200))
bar = ''.join('#' if c > len(values) / 30 else ('.' if c else ' ') for c in counts)
print(f'{name:<6} |{bar}|')
print(' ' + '50' + ' ' * 11 + '200')
Open this example in the editor and change it. Nothing you type is saved.
Three distributions with similar means and standard deviations: one hump, two humps, and one hump plus a small separate group. Only the third row of the text histogram reveals the hundred records at 180, which in real data is usually the interesting finding - a different population that got mixed in.
Always plot the distribution before quoting a summary of it. describe() is a check, not a conclusion.
Variation matters as much as level
import numpy as np
import pandas as pd
rng = np.random.default_rng(29)
steady = rng.normal(100, 4, 200)
erratic = rng.normal(100, 28, 200)
frame = pd.DataFrame({
'mean': [steady.mean(), erratic.mean()],
'std': [steady.std(), erratic.std()],
'cv': [steady.std() / steady.mean(), erratic.std() / erratic.mean()],
'p5': [np.percentile(steady, 5), np.percentile(erratic, 5)],
'p95': [np.percentile(steady, 95), np.percentile(erratic, 95)],
}, index=['steady', 'erratic']).round(2)
print(frame)
print()
print('Same average delivery time. One supplier you can plan around,')
print('one you cannot - and the mean cannot tell them apart.')
Open this example in the editor and change it. Nothing you type is saved.
The coefficient of variation - standard deviation over mean - makes variability comparable across things measured in different units. A CV of 0.04 is a process under control; 0.28 is one that is not.
For a skewed distribution the standard deviation is as misleading as the mean, and the interquartile range is the better measure:
import numpy as np
rng = np.random.default_rng(31)
skewed = rng.lognormal(3, 1.2, 5000)
print(f'std {skewed.std():.1f}')
print(f'iqr {np.percentile(skewed, 75) - np.percentile(skewed, 25):.1f}')
print(f'mad {np.median(np.abs(skewed - np.median(skewed))):.1f}')
Open this example in the editor and change it. Nothing you type is saved.
A worked example
import numpy as np
import pandas as pd
rng = np.random.default_rng(seed=97)
# Support ticket resolution times, in hours, across three teams.
# Two teams have similar means and very different experiences.
n = 4000
teams = []
for name, base, tail_share, tail_scale in [
('alpha', 4.0, 0.02, 60), # consistent, tiny tail
('bravo', 3.2, 0.14, 90), # faster typical, much worse tail
('charlie', 5.5, 0.03, 40), # slower typical, controlled tail
]:
count = n // 3
tail = rng.random(count) < tail_share
hours = np.where(tail,
rng.gamma(2.0, tail_scale, count),
rng.gamma(2.0, base, count))
teams.append(pd.DataFrame({'team': name, 'hours': hours}))
tickets = pd.concat(teams, ignore_index=True)
# 1. What the mean says.
print('by mean alone:')
print(tickets.groupby('team')['hours'].mean().round(1).to_string())
print()
# 2. What the percentiles say.
summary = tickets.groupby('team')['hours'].agg(
n='size',
mean='mean',
p50=lambda s: s.quantile(0.50),
p90=lambda s: s.quantile(0.90),
p95=lambda s: s.quantile(0.95),
p99=lambda s: s.quantile(0.99),
worst='max',
).round(1)
print(summary.to_string())
print()
# 3. The ratio that reveals the tail.
summary['mean_over_p50'] = (summary['mean'] / summary['p50']).round(2)
summary['p99_over_p50'] = (summary['p99'] / summary['p50']).round(1)
print(summary[['mean', 'p50', 'mean_over_p50', 'p99_over_p50']].to_string())
print()
print('bravo has the BEST median and by far the worst tail.')
print('The mean calls it the worst team; the median calls it the best.')
print('Both are true, and the ratio above is the explanation.')
print()
# 4. Where the mean actually sits in each distribution.
for team, group in tickets.groupby('team'):
above = (group['hours'] > group['hours'].mean()).mean()
print(f'{team:<8} mean {group["hours"].mean():>5.1f}h sits at the '
f'{100 - above * 100:.0f}th percentile '
f'({above:.1%} of tickets are slower)')
print('A number most tickets beat is not a typical ticket.')
print()
# 5. Variability, comparable across teams.
spread = tickets.groupby('team')['hours'].agg(
std='std',
iqr=lambda s: s.quantile(0.75) - s.quantile(0.25),
cv=lambda s: s.std() / s.mean(),
).round(2)
print(spread.to_string())
print()
# 6. The shape, as text, because the numbers above do not show it.
print('distribution of resolution time (0 to 60 hours):')
for team, group in tickets.groupby('team'):
counts, _ = np.histogram(group['hours'], bins=30, range=(0, 60))
peak = counts.max()
bar = ''.join('#' if c > peak * 0.30 else ('+' if c > peak * 0.08 else
('.' if c else ' ')) for c in counts)
over = (group['hours'] > 60).sum()
print(f' {team:<8} |{bar}| {over} over 60h')
print()
# 7. The sentence, with the right number in it.
best_median = summary['p50'].idxmin()
best_p95 = summary['p95'].idxmin()
print(f'Fastest typical ticket: {best_median} (p50 {summary.loc[best_median, "p50"]}h).')
print(f'Most reliable: {best_p95} (p95 {summary.loc[best_p95, "p95"]}h).')
print('These are different teams, which is the whole finding.')
Open this example in the editor and change it. Nothing you type is saved.
The finding in that example is only visible because of the percentiles. The mean calls bravo the worst team by a wide margin. The median calls it the best. Both are correct: bravo resolves the typical ticket fastest and has a tail four times longer than anyone else's, and the customers in that tail are the ones who escalate. A report with only the mean would have missed the good news; a report with only the median would have missed the bad.
The fourth step is worth a second look too. In every one of these distributions the mean sits well above the median - between the 68th and 85th percentile - so the "average" resolution time is one that most tickets beat. That is not a quirk of this data; it is what a right-skewed distribution does, and it is why a mean presented as "typical" overstates the typical experience for the majority while understating it badly for the tail.
The practical rule
For any distribution you have not looked at:
- Plot it. A histogram takes one line and answers the shape question.
- Compare mean and median. If they differ by more than a few percent, the mean is not typical.
- Report p50, p90 and p99 for anything a person experiences - latency, wait time, delivery.
- Report the median and a range for anything about money per person.
- Publish the count alongside, so a reader can judge how much to trust it.
- Never report a mean alone for a skewed quantity. It describes nobody.
When it goes wrong
| Symptom | Cause |
|---|---|
| The "average customer" does not exist | A skewed distribution summarised by its mean |
| One record moved the headline number | Means are not robust; use the median |
| The system looks fast and users complain | The mean hides the tail; look at p95 and p99 |
Two groups look identical in describe() |
Different shapes, same summary; plot them |
| A standard deviation is larger than the mean | Heavy skew; use the IQR instead |
| A small segment looks dramatic | Too few records; publish the count |
| A percentage with no denominator | "60% improvement" of what, from what |
| An average of averages is wrong | Weight by the group sizes |
That last one deserves a demonstration, because it is the most common arithmetic error in reporting:
import numpy as np
import pandas as pd
groups = pd.DataFrame({
'team': ['alpha', 'bravo'],
'tickets': [900, 100],
'mean_hours': [4.0, 20.0],
})
naive = groups['mean_hours'].mean()
weighted = np.average(groups['mean_hours'], weights=groups['tickets'])
print(f'average of the averages: {naive:.1f} hours')
print(f'weighted by volume: {weighted:.1f} hours')
Open this example in the editor and change it. Nothing you type is saved.
A check you can run
Take the last average you published and compute the median beside it.
If they differ by more than about 5%, the mean was the wrong number. Then compute where the mean falls as a percentile: for a right-skewed quantity it is routinely the 70th or higher, meaning most of your population is on the better side of the number you published, and a long tail is on the far worse side.
Either sentence changes the conversation: "most tickets beat our published average, and the 5% that do not take ten times longer" is a far more useful statement than a single mean, and it is two lines of code away.