Field note · November 22, 2024

Five minutes with humidity_pct in sensor-telemetry

8,000 rows, 260 distinct values, and the one fact about humidity_pct that changes how you query it.

1 min read ·Data quality ·quality practice

Someone asked what is in humidity_pct on sensor-telemetry, and the honest answer took one query.

8,000 rows, no nulls, 260 distinct values. Ambient relative humidity.

The five-number version: min 24.90, p10 35.50, median 40.90, p90 46.30, max 55.90. The mean is 40.91.

The p10 to p90 band — 35.50 to 46.30 — is where ordinary rows live, and it is the pair worth quoting when somebody asks what to expect. Min and max describe the two strangest rows in the table and nothing else; they are useful for spotting impossible values and misleading for everything else.

sql
select
  count(*)                       as rows,
  count(humidity_pct)            as present,
  count(*) - count(humidity_pct) as nulls,
  count(distinct humidity_pct)   as distinct_values
from sensor_telemetry;

Nothing dramatic: the mean and median are within 0.0% of each other, so an average is a fair summary. That is worth confirming rather than assuming — it is not true of most money columns.

Where this matters: a symmetric column is one you can average, threshold and chart without hedging, which makes it unusually cheap to work with. Knowing which of your columns are like this and which are not is most of knowing when to be careful.

The grain is one row per sensor reading, which is the context every one of those numbers depends on. None of them survive a change of grain, which is why "profile the column" and "profile the table" are the same job. Full schema, and the CSV, on the dataset page.