Blog, page 3

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Jun 21, 2026

Correction: Gaps and islands

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Analytics engineering

Jun 20, 2026

When should I not build a model?

When a rule gets you most of the way, when you cannot get labels, when nobody will act on the prediction, or when being wrong is expensive and unexpla

modelling practice Machine learning

Jun 19, 2026

The build takes under two seconds

Full rebuild: generate fourteen datasets from scratch, render every page, write the feeds, verify every internal link. Under two seconds on a laptop.

engineering performance Data platform

Jun 18, 2026

Charting flight_no by origin

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Jun 17, 2026

3.0%, plus or minus what?

A point estimate of 3.0%, the interval around it, and what a tighter one would cost.

statistics Statistics

Jun 16, 2026

If we had to cut the course to ten lessons

An argument we had, and the list we landed on.

practice Experimentation

Jun 15, 2026

Why Chart Builder works the way it does

Point it at any dataset, choose an x, a y, and a grouping, and get a chart that follows the rules — one axis, direct labels, a real legend, and a table view underneath.

visualization Visualization

Jun 14, 2026

Data job postings, in one paragraph

Thirty-five hundred data job ads with titles, seniority, location, remote policy, salary bands, and a pipe-delimited skills column that needs parsing before it is useful..

quality practice Data quality

Jun 13, 2026

Reader question — Reading a result without fooling yourself

The experiment finished. Here is the order to look at things in, and the specific ways a real-looking result turns out not to be.

practice Experimentation

Jun 12, 2026

Deduplicate by rank, in short

Keep the best row per key with a window function instead of a self-join or a GROUP BY that loses columns.

sql Analytics engineering

Jun 11, 2026

The time coverage of sensor-telemetry

59 populated days across a 59-day span, and what that allows you to compute.

engineering quality Data engineering

Jun 10, 2026

flight-delays.dest: 14 values, 22.3% in the top three

14 values with 7.5% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Jun 9, 2026

Is posting_id + country the grain of data-job-postings?

3,500 distinct combinations against 3,500 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jun 8, 2026

A running total that does not lie about ties

running total against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Jun 7, 2026

The total came out 1.0× too high

1.0 rows per key on one table, and the totals that quietly inherit the multiplier.

sql quality Analytics engineering

Jun 6, 2026

What device actually contains in ab-test-checkout

6,000 rows, 3 distinct values, and the one fact about device that changes how you query it.

quality practice Data quality

Jun 5, 2026

21.8% of salary_min_usd is missing

762 blanks in one column, and the one-line check that decides whether dropping them is free.

quality statistics Statistics

Jun 4, 2026

arr_delay_min: mean 7, median 2

A 227.8% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

Jun 3, 2026

Zero is not a small number

51.3% of one column is exactly zero. Whether that is a measurement or an absence changes every aggregate built on it.

quality statistics Data quality

Jun 2, 2026

Reading 3 columns instead of 8

Column pruning times partition pruning on a real schema, and the function call that quietly defeats both.

performance warehousing Data platform

Jun 1, 2026

Before you clip the outliers

A max 1.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

May 31, 2026

dep_delay_min by carrier: a 85% spread

Real group means for one column across 6 segments, and what the pooled average conceals.

statistics practice Analytics practice

May 30, 2026

A essentially none correlation, and what it is not

A measured correlation of 0.08 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

May 29, 2026

Correction: Idempotent window replace

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Data engineering