Field note · June 27, 2024
The overall average hides 4 different numbers
Real group means for one column across 4 segments, and what the pooled average conceals.
plan splits saas-subscriptions into 4 groups. Here is what mrr_usd looks like inside each.
enterprise— mean 4,282, median 4,361 (150 rows)business— mean 693, median 655 (442 rows)team— mean 166, median 160 (800 rows)starter— mean 31, median 27 (1,108 rows)
Top to bottom that is 4,282 against 31, a spread of 13,612.2%. The pooled average is 447.
select plan,
count(*) as rows,
round(avg(mrr_usd)::numeric, 2) as mean,
percentile_cont(0.5) within group (order by mrr_usd) as median
from saas_subscriptions
group by 1
order by mean desc;A spread that wide means the pooled number is not a summary, it is an artefact of the mix. Change the proportion of enterprise rows and the overall average moves without any individual group changing at all — which is how a metric goes up while every segment goes down.
Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.
This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.