Field note · July 18, 2024
release_year by genre: a 0% spread
Real group means for one column across 8 segments, and what the pooled average conceals.
Breaking release_year down by genre on movie-ratings, because the headline average is 1,999 and no segment is actually there.
romance— mean 2,003, median 2,012 (1,950 rows)comedy— mean 2,001, median 2,000 (1,330 rows)documentary— mean 1,999, median 1,998 (675 rows)thriller— mean 1,999, median 2,008 (1,258 rows)drama— mean 1,999, median 1,996 (843 rows)horror— mean 1,997, median 1,991 (832 rows)
Top to bottom that is 2,003 against 1,996, a spread of 0.3%. The pooled average is 1,999.
select genre,
count(*) as rows,
round(avg(release_year)::numeric, 2) as mean,
percentile_cont(0.5) within group (order by release_year) as median
from movie_ratings
group by 1
order by mean desc;The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.
Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.
This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.