Field note · August 13, 2023

distance_km by borough: a 331% spread

Real group means for one column across 5 segments, and what the pooled average conceals.

1 min read ·Analytics practice ·statistics practice

Breaking distance_km down by borough on ride-hail-trips, because the headline average is 4.83 and no segment is actually there.

  • Airport — mean 14.77, median 13.59 (965 rows)
  • Uptown — mean 3.51, median 2.90 (1,453 rows)
  • Downtown — mean 3.47, median 2.87 (2,399 rows)
  • Midtown — mean 3.47, median 2.90 (2,034 rows)
  • Harbour — mean 3.43, median 2.89 (1,149 rows)

Top to bottom that is 14.77 against 3.43, a spread of 331.0%. The pooled average is 4.83.

sql
select borough,
  count(*)                                                 as rows,
  round(avg(distance_km)::numeric, 2)                      as mean,
  percentile_cont(0.5) within group (order by distance_km) as median
from ride_hail_trips
group by 1
order by mean desc;

A spread that wide means the pooled number is not a summary, it is an artefact of the mix. Change the proportion of Airport rows and the overall average moves without any individual group changing at all — which is how a metric goes up while every segment goes down.

Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.

This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.