Field note · September 15, 2023
The overall average hides 30 different numbers
Real group means for one column across 30 segments, and what the pooled average conceals.
country splits world-indicators into 30 groups. Here is what gdp_per_capita_usd looks like inside each.
Xanthe— mean 79,795, median 74,032 (24 rows)Belmara— mean 59,947, median 55,588 (24 rows)Lumeria— mean 58,947, median 58,216 (24 rows)Cairnvale— mean 58,851, median 59,382 (24 rows)Quorra— mean 55,207, median 57,046 (24 rows)Novastan— mean 53,635, median 48,660 (24 rows)
Top to bottom that is 79,795 against 2,117, a spread of 3,669.9%. The pooled average is 26,959.
select country,
count(*) as rows,
round(avg(gdp_per_capita_usd)::numeric, 2) as mean,
percentile_cont(0.5) within group (order by gdp_per_capita_usd) as median
from world_indicators
group by 1
order by mean desc;A spread that wide means the pooled number is not a summary, it is an artefact of the mix. Change the proportion of Xanthe rows and the overall average moves without any individual group changing at all — which is how a metric goes up while every segment goes down.
Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.
This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.