Field note · May 28, 2023

Five minutes with reading_date in city-air-quality

5,480 rows, 1,370 distinct values, and the one fact about reading_date that changes how you query it.

1 min read ·Data quality ·quality practice

Someone asked what is in reading_date on city-air-quality, and the honest answer took one query.

5,480 rows, no nulls, 1,370 distinct values. Calendar date.

It runs from 2023-01-01 to 2026-10-01, covering 1,370 distinct days.

It is a date rather than a timestamp, so there is no time zone to get wrong here. That is worth noticing, because the moment a column like this gains a time component, every daily aggregate silently shifts for anyone not in UTC.

sql
select
  count(*)                       as rows,
  count(reading_date)            as present,
  count(*) - count(reading_date) as nulls,
  count(distinct reading_date)   as distinct_values
from city_air_quality;

Check the range before filtering on it. Half the "the dashboard is empty" reports we have seen are a date filter outside the data's actual range, and the query is not wrong so nothing errors.

Where this bites: 1,370 populated days is what any window function over this column has to work with. A seven-day lag counts rows, not days — so if a day is missing, lag(7) quietly compares against eight days ago and the week-over-week number is wrong in a way that looks plausible.

The grain is one row per city per day, which is the context every one of those numbers depends on. None of them survive a change of grain, which is why "profile the column" and "profile the table" are the same job. Full schema, and the CSV, on the dataset page.