Field note · July 16, 2023
The overall average hides 30 different numbers
Real group means for one column across 30 segments, and what the pooled average conceals.
country splits world-indicators into 30 groups. Here is what year looks like inside each.
Avalonia— mean 2,012, median 2,012 (24 rows)Brasilia Nova— mean 2,012, median 2,012 (24 rows)Corvina— mean 2,012, median 2,012 (24 rows)Dunmara— mean 2,012, median 2,012 (24 rows)Eldoria— mean 2,012, median 2,012 (24 rows)Ferrovia— mean 2,012, median 2,012 (24 rows)
Top to bottom that is 2,012 against 2,012. The pooled average is 2,012.
select country,
count(*) as rows,
round(avg(year)::numeric, 2) as mean,
percentile_cont(0.5) within group (order by year) as median
from world_indicators
group by 1
order by mean desc;The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.
Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.
This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.