Field note · September 8, 2024

The overall average hides 6 different numbers

Real group means for one column across 6 segments, and what the pooled average conceals.

1 min read ·Analytics practice ·statistics practice

site splits clinical-trial into 6 groups. Here is what bmi looks like inside each.

  • site-3 — mean 27.66, median 27.50 (148 rows)
  • site-1 — mean 27.34, median 27.50 (142 rows)
  • site-2 — mean 27.19, median 26.85 (174 rows)
  • site-5 — mean 27.01, median 27.10 (138 rows)
  • site-6 — mean 26.82, median 27.15 (146 rows)
  • site-4 — mean 26.76, median 26.40 (152 rows)

Top to bottom that is 27.66 against 26.76, a spread of 3.4%. The pooled average is 27.13.

sql
select site,
  count(*)                                         as rows,
  round(avg(bmi)::numeric, 2)                      as mean,
  percentile_cont(0.5) within group (order by bmi) as median
from clinical_trial
group by 1
order by mean desc;

The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.

Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.

This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.