What dep_delay_min actually contains in flight-delays

Working through flight-delays again. dep_delay_min is the column people trip over, so here is what it actually looks like.

8,000 rows, 167 nulls (2.1%), 251 distinct values. Departure delay in minutes; negative means early. Null if cancelled.

The five-number version: min -47, p10 -14, median 5, p90 28, max 478. The mean is 10.

The p10 to p90 band — -14 to 28 — is where ordinary rows live, and it is the pair worth quoting when somebody asks what to expect. Min and max describe the two strangest rows in the table and nothing else; they are useful for spotting impossible values and misleading for everything else.

167 rows have no value at all here, which is 2.1% of the table. That is enough to move an aggregate and small enough that nobody notices it doing so — every average above is computed over 7,833 rows, not 8,000, and the two denominators produce different answers.

select
  count(*)                        as rows,
  count(dep_delay_min)            as present,
  count(*) - count(dep_delay_min) as nulls,
  count(distinct dep_delay_min)   as distinct_values
from flight_delays;

The mean sits 99% above the median, which is the whole story: anything that reports the average of dep_delay_min is reporting a number most rows are below. Put the median next to it or drop the mean.

Where this bites: a filter like dep_delay_min > 10 reads as "above average" and selects a minority of rows — a smaller minority than the phrase suggests to whoever asked for it. If the request was "the typical ones", the threshold they meant was 5.

The grain is one row per scheduled flight, which is the context every one of those numbers depends on. None of them survive a change of grain, which is why "profile the column" and "profile the table" are the same job. Full schema, and the CSV, on the dataset page.