Field note · January 12, 2024
The overall average hides 14 different numbers
Real group means for one column across 14 segments, and what the pooled average conceals.
dest splits flight-delays into 14 groups. Here is what dep_delay_min looks like inside each.
MCO— mean 11, median 5 (573 rows)DFW— mean 11, median 5 (569 rows)ORD— mean 11, median 6 (550 rows)JFK— mean 11, median 5 (585 rows)SFO— mean 11, median 6 (601 rows)IAH— mean 10, median 5 (578 rows)
Top to bottom that is 11 against 8, a spread of 38.4%. The pooled average is 10.
select dest,
count(*) as rows,
round(avg(dep_delay_min)::numeric, 2) as mean,
percentile_cont(0.5) within group (order by dep_delay_min) as median
from flight_delays
group by 1
order by mean desc;The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.
Notice the mean and median columns disagree most in MCO, where the mean sits well above the median. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.
This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.