Field note · September 29, 2023
The overall average hides 4 different numbers
Real group means for one column across 4 segments, and what the pooled average conceals.
plan splits saas-subscriptions into 4 groups. Here is what seats_now looks like inside each.
enterprise— mean 642, median 654 (150 rows)business— mean 79, median 76 (442 rows)team— mean 16, median 16 (800 rows)starter— mean 3, median 2 (1,108 rows)
Top to bottom that is 642 against 3, a spread of 25,266.7%. The pooled average is 59.
select plan,
count(*) as rows,
round(avg(seats_now)::numeric, 2) as mean,
percentile_cont(0.5) within group (order by seats_now) as median
from saas_subscriptions
group by 1
order by mean desc;A spread that wide means the pooled number is not a summary, it is an artefact of the mix. Change the proportion of enterprise rows and the overall average moves without any individual group changing at all — which is how a metric goes up while every segment goes down.
Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.
This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.