Blog

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,244 posts. , or browse by year and tag below.

Sep 21, 2026

Correction: Merge with a recency guard

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Data engineering

Sep 20, 2026

Why does my query cost so much?

Almost always because it read data it did not need.

performance warehousing Data platform

Sep 19, 2026

The link checker that runs on every build

Every build walks the generated HTML and verifies that each internal href and src resolves to something on disk. It is about forty lines and it has ca

engineering quality Data platform

Sep 18, 2026

9.7%, plus or minus what?

A point estimate of 9.7%, the interval around it, and what a tighter one would cost.

statistics Statistics

Sep 17, 2026

Chart Builder works offline

Point it at any dataset, choose an x, a y, and a grouping, and get a chart that follows the rules — one axis, direct labels, a real legend, and a table view underneath.

visualization Visualization

Sep 16, 2026

An exercise on world-indicators

A tidy panel of thirty countries over twenty-four years with GDP per capita, life expectancy, population, CO₂ and internet penetration — small enough to hold in your head, rich enough for real modelling..

quality practice Data quality

Sep 15, 2026

What actually changed in this field in three years

We started writing in 2023. Here is what dated, what did not, and what we had to rewrite.

practice Analytics engineering

Sep 14, 2026

The category that eats the chart

3 values with 45.5% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Sep 13, 2026

Course note — Rows, grain, and the shape of a dataset

Nearly every wrong number in a career traces back to someone not knowing what one row of their table represents.

practice Analytics practice

Sep 12, 2026

Deterministic surrogate key, in short

Derive the key from the content instead of an auto-increment, so reprocessing the same row produces the same key.

pipelines modelling Data engineering

Sep 11, 2026

resolved_at runs 2023-01-01 to 2024-02-04

400 populated days across a 400-day span, and what that allows you to compute.

engineering quality Data engineering

Sep 10, 2026

Keeping one row per key in data-job-postings

deduplicate by rank against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Sep 9, 2026

movie_id is not unique, and the join knows

34.6 rows per key on one table, and the totals that quietly inherit the multiplier.

sql quality Analytics engineering

Sep 8, 2026

mrr_usd: mean 447, median 88

A 410.2% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

Sep 7, 2026

Counting rows is not testing grain

8,000 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Sep 6, 2026

0.7% of signup_date is missing

7 blanks in one column, and the one-line check that decides whether dropping them is free.

quality statistics Statistics

Sep 5, 2026

Zero is not a small number

2.3% of one column is exactly zero. Whether that is a measurement or an absence changes every aggregate built on it.

quality statistics Data quality

Sep 4, 2026

What industry actually contains in saas-subscriptions

2,500 rows, 8 distinct values, and the one fact about industry that changes how you query it.

quality practice Data quality

Sep 3, 2026

The overall average hides 10 different numbers

Real group means for one column across 10 segments, and what the pooled average conceals.

statistics practice Analytics practice

Sep 2, 2026

A essentially none correlation, and what it is not

A measured correlation of -0.02 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Sep 1, 2026

The arithmetic behind a cheap query

Column pruning times partition pruning on a real schema, and the function call that quietly defeats both.

performance warehousing Data platform

Aug 31, 2026

The top of vibration_mm_s is 3.2× the 99th percentile

A max 3.2× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Aug 30, 2026

What 3,500 job postings say about salary_min_usd

One column of a postings dataset, and what the shape of it means for anyone reading listings.

career practice Analytics practice

Aug 29, 2026

Charting sessions by variant

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization