Field note · July 5, 2023
duration_min by payment_type: a 3% spread
Real group means for one column across 3 segments, and what the pooled average conceals.
Breaking duration_min down by payment_type on ride-hail-trips, because the headline average is 20.11 and no segment is actually there.
card— mean 20.29, median 14.40 (4,983 rows)cash— mean 19.94, median 14.20 (892 rows)wallet— mean 19.75, median 14.40 (2,125 rows)
Top to bottom that is 20.29 against 19.75, a spread of 2.7%. The pooled average is 20.11.
select payment_type,
count(*) as rows,
round(avg(duration_min)::numeric, 2) as mean,
percentile_cont(0.5) within group (order by duration_min) as median
from ride_hail_trips
group by 1
order by mean desc;The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.
Notice the mean and median columns disagree most in card, where the mean sits well above the median. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.
This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.