Field note · July 18, 2024

release_year by genre: a 0% spread

Real group means for one column across 8 segments, and what the pooled average conceals.

1 min read ·Analytics practice ·statistics practice

Breaking release_year down by genre on movie-ratings, because the headline average is 1,999 and no segment is actually there.

  • romance — mean 2,003, median 2,012 (1,950 rows)
  • comedy — mean 2,001, median 2,000 (1,330 rows)
  • documentary — mean 1,999, median 1,998 (675 rows)
  • thriller — mean 1,999, median 2,008 (1,258 rows)
  • drama — mean 1,999, median 1,996 (843 rows)
  • horror — mean 1,997, median 1,991 (832 rows)

Top to bottom that is 2,003 against 1,996, a spread of 0.3%. The pooled average is 1,999.

sql
select genre,
  count(*)                                                  as rows,
  round(avg(release_year)::numeric, 2)                      as mean,
  percentile_cont(0.5) within group (order by release_year) as median
from movie_ratings
group by 1
order by mean desc;

The groups are close enough that the pooled average is a fair summary. That is worth confirming rather than assuming: the check costs one query and the failure mode is invisible.

Notice the mean and median columns disagree only slightly here. Always compute both in the group-by. The comparison between them per segment is free and tells you whether you are looking at a level difference or a tail difference.

This is the setup for Simpson's paradox — the case where every segment moves one way and the total moves the other.