Field note · July 29, 2025

The CLT does not always save you

A tool for watching sampling distributions, and the picture that changed how I teach it.

1 min read ·Statistics ·statistics

The distribution lab is up. Pick a population, pick a sample size, and watch the sampling distribution of the mean.

The reason I wanted it: "n > 30 so the central limit theorem applies" is folklore, it is taught as a rule, and it is wrong often enough to matter.

The CLT says the sampling distribution of the mean approaches normal as n grows, whatever the population's shape. What gets dropped in the retelling is how fast, which depends entirely on the population's skew.

Try it. Set the population to lognormal with sigma 0.8. At n = 5 the sampling distribution is visibly skewed. At n = 30 — the folklore threshold — it is still visibly skewed. It takes a few hundred to look normal.

Then try Pareto with alpha 1.4. It does not converge at any n you can set, because a single observation can dominate the sum. That is not a slow convergence; it is a distribution where the sample mean is unstable at any realistic sample size.

Both of those shapes describe real business data. Revenue per user is lognormal-ish at best. Insurance claims and enterprise deal sizes have Pareto-ish tails.

The practical consequences:

  • A t-test on revenue per user at n = 200 is not doing what you think.
  • Bootstrap instead. It makes no distributional assumption.
  • Or decompose: analyse conversion rate and value-among-converters separately. Two clean questions instead of one messy one.

The tool draws the mean and median on every histogram, and watching them separate as you increase skew is the fastest way I know to build the intuition. It is seeded, so any picture you generate is reproducible.