Field note · August 7, 2023

year by country: a wide spread

Real group means for one column across 30 segments, and what the pooled average conceals.

1 min read ·Analytics practice ·statistics practice

Breaking year down by country on world-indicators, because the headline average is 2,012 and no segment is actually there.

  • Avalonia — mean 2,012, median 2,012 (24 rows)
  • Brasilia Nova — mean 2,012, median 2,012 (24 rows)
  • Corvina — mean 2,012, median 2,012 (24 rows)
  • Dunmara — mean 2,012, median 2,012 (24 rows)
  • Eldoria — mean 2,012, median 2,012 (24 rows)
  • Ferrovia — mean 2,012, median 2,012 (24 rows)

Top to bottom that is 2,012 against 2,012. The pooled average is 2,012.

sql
select country,
  count(*)                                          as rows,
  round(avg(year)::numeric, 2)                      as mean,
  percentile_cont(0.5) within group (order by year) as median
from world_indicators
group by 1
order by mean desc;

The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.

Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.

This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.