Field note · March 21, 2025

Counting rows is not testing grain

3,780 distinct combinations against 4,000 rows, and what that answers about the table's key.

2 min read ·Analytics engineering ·sql quality warehousing

Every table has a sentence describing what one row is. On support-tickets it is "one row per ticket". Here is the test that the data agrees.

resolved_at on its own has 3,776 distinct values across 4,000 rows, so it repeats — an average of 1.1 rows per value. Adding queue gives 3,780 distinct combinations, still short of the 4,000 rows.

sql
select resolved_at, queue, count(*) as n
from support_tickets
group by 1, 2
having count(*) > 1
order by n desc
limit 10;

That query returns rows, so the pair is not the grain — there is at least one more column in the key. Joining on it as if it were unique multiplies rows on the other side, and the failure shows up as a total that is too high rather than as an error.

The having count(*) > 1 formulation matters more than it looks. A bare count(distinct ...) tells you whether the key holds; this tells you which rows break it, which is the difference between knowing you have a problem and being able to fix it. Sort by n desc and the worst offender is the first row — usually enough to identify the cause without another query.

Two things this test does not tell you. It does not say whether the duplicates are wrong: some tables legitimately carry a history, and the fix there is to add the version column to the key rather than to delete rows. And it says nothing about the future — a key that holds across 4,000 rows today can break on the next load, which is the argument for running it on a schedule rather than once during exploration.

Until the real key is known, treat every aggregate over this table as provisional. The safe move is to aggregate to a grain you have verified before joining anything to it — that converts an unknown multiplier into an explicit one.

A grain claim you have not tested is a comment, not a constraint. It takes one query to convert it, and the test costs nothing to run nightly. The pattern has the version we ship, and Rows, grain, and the shape of a dataset is the long form.

AE

Analytics engineering

Turning warehouses full of raw tables into models an analyst can trust without asking anyone.

Dimensional modelling, grain, slowly changing dimensions, metric definitions, and the SQL that survives production. Most "we need a data scientist" problems are "we need one correct table" problems.

Related

All field notes    2025 archive