Field note · August 28, 2024

The grain uniqueness test, in short

One test per model, on the key that defines its grain. The cheapest quality control available and the one that catches the worst bugs.

1 min read ·Data quality ·quality

Someone asked why The grain uniqueness test is written the way it is. Fair question.

One test per model, on the key that defines its grain. The cheapest quality control available and the one that catches the worst bugs.

What makes it a pattern rather than a tip is that the wrong version is the one you write naturally. It reads correctly, it runs, and it returns something. The failure is in the result, not in the execution — which means the only defence is recognising the shape before you are in it.

Two datasets on this site have the shape built in: retail-orders and support-tickets. Both are small enough to run the broken version, see the number, then run the corrected one and see it change.

It lives under moving because it is about data in transit: reruns, backfills, and the assumption that yesterday only ever arrives once.

The long-form treatment is in the course (testing-data-like-code, rows-grain-shape); the pattern page is the version to read at 4pm with a query open.

Whether you use our version matters much less than having a version you did not re-derive under time pressure. That is what a pattern library is for.