Field note · January 26, 2024

sched_dep_hour by dest: a 8% spread

Real group means for one column across 14 segments, and what the pooled average conceals.

1 min read ·Analytics practice ·statistics practice

Breaking sched_dep_hour down by dest on flight-delays, because the headline average is 13 and no segment is actually there.

  • BOS — mean 13, median 14 (597 rows)
  • ORD — mean 13, median 13 (550 rows)
  • SEA — mean 13, median 13 (557 rows)
  • PHX — mean 13, median 13 (578 rows)
  • DFW — mean 13, median 13 (569 rows)
  • DTW — mean 13, median 13 (548 rows)

Top to bottom that is 13 against 12, a spread of 7.7%. The pooled average is 13.

sql
select dest,
  count(*)                                                    as rows,
  round(avg(sched_dep_hour)::numeric, 2)                      as mean,
  percentile_cont(0.5) within group (order by sched_dep_hour) as median
from flight_delays
group by 1
order by mean desc;

The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.

Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.

This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.