Field note · October 8, 2023

flight_no by carrier: a 3% spread

Real group means for one column across 6 segments, and what the pooled average conceals.

1 min read ·Analytics practice ·statistics practice

Breaking flight_no down by carrier on flight-delays, because the headline average is 3,560 and no segment is actually there.

  • B6 — mean 3,606, median 3,604 (766 rows)
  • DL — mean 3,606, median 3,613 (1,684 rows)
  • AA — mean 3,588, median 3,651 (1,751 rows)
  • UA — mean 3,553, median 3,501 (1,422 rows)
  • AS — mean 3,520, median 3,441 (725 rows)
  • WN — mean 3,487, median 3,498 (1,652 rows)

Top to bottom that is 3,606 against 3,487, a spread of 3.4%. The pooled average is 3,560.

sql
select carrier,
  count(*)                                               as rows,
  round(avg(flight_no)::numeric, 2)                      as mean,
  percentile_cont(0.5) within group (order by flight_no) as median
from flight_delays
group by 1
order by mean desc;

The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.

Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.

This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.