Field note · May 15, 2023

The overall average hides 8 different numbers

Real group means for one column across 8 segments, and what the pooled average conceals.

1 min read ·Analytics practice ·statistics practice

country splits retail-orders into 8 groups. Here is what discount_pct looks like inside each.

  • CA — mean 0.10, median 0.00 (715 rows)
  • FR — mean 0.10, median 0.05 (522 rows)
  • US — mean 0.10, median 0.00 (2,655 rows)
  • DE — mean 0.10, median 0.00 (738 rows)
  • BR — mean 0.10, median 0.00 (367 rows)
  • JP — mean 0.09, median 0.00 (587 rows)

Top to bottom that is 0.10 against 0.09, a spread of 14.4%. The pooled average is 0.10.

sql
select country,
  count(*)                                                  as rows,
  round(avg(discount_pct)::numeric, 2)                      as mean,
  percentile_cont(0.5) within group (order by discount_pct) as median
from retail_orders
group by 1
order by mean desc;

The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.

Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.

This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.