Field note · September 15, 2023

The overall average hides 30 different numbers

Real group means for one column across 30 segments, and what the pooled average conceals.

1 min read ·Analytics practice ·statistics practice

country splits world-indicators into 30 groups. Here is what gdp_per_capita_usd looks like inside each.

  • Xanthe — mean 79,795, median 74,032 (24 rows)
  • Belmara — mean 59,947, median 55,588 (24 rows)
  • Lumeria — mean 58,947, median 58,216 (24 rows)
  • Cairnvale — mean 58,851, median 59,382 (24 rows)
  • Quorra — mean 55,207, median 57,046 (24 rows)
  • Novastan — mean 53,635, median 48,660 (24 rows)

Top to bottom that is 79,795 against 2,117, a spread of 3,669.9%. The pooled average is 26,959.

sql
select country,
  count(*)                                                        as rows,
  round(avg(gdp_per_capita_usd)::numeric, 2)                      as mean,
  percentile_cont(0.5) within group (order by gdp_per_capita_usd) as median
from world_indicators
group by 1
order by mean desc;

A spread that wide means the pooled number is not a summary, it is an artefact of the mix. Change the proportion of Xanthe rows and the overall average moves without any individual group changing at all — which is how a metric goes up while every segment goes down.

Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.

This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.