Blog, page 8

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Feb 21, 2026

Finding the gaps in saas-subscriptions

gaps and islands against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Feb 20, 2026

Is user_id + genre the grain of movie-ratings?

5,354 distinct combinations against 9,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Feb 19, 2026

The average arr delay min is not the typical one

A 227.8% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

Feb 18, 2026

Reading borough before trusting it in ride-hail-trips

8,000 rows, 5 distinct values, and the one fact about borough that changes how you query it.

quality practice Data quality

Feb 17, 2026

Reading 3 columns instead of 11

Column pruning times partition pruning on a real schema, and the function call that quietly defeats both.

performance warehousing Data platform

Feb 16, 2026

The top of revenue_usd is 2.8× the 99th percentile

A max 2.8× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Feb 15, 2026

population by country: a 4,739% spread

Real group means for one column across 30 segments, and what the pooled average conceals.

statistics practice Analytics practice

Feb 14, 2026

A weak correlation, and what it is not

A measured correlation of -0.36 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Feb 13, 2026

Correction: Sessionisation

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Analytics engineering

Feb 12, 2026

Charting unit_price_usd by channel

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Feb 11, 2026

2.1%, plus or minus what?

A point estimate of 2.1%, the interval around it, and what a tighter one would cost.

statistics Statistics

Feb 10, 2026

This whole site is 900 lines of Node and one dependency

Why we wrote our own generator instead of using a framework, and what that bought.

engineering Data engineering

Feb 9, 2026

A five-minute use for Chart Builder

Point it at any dataset, choose an x, a y, and a grouping, and get a chart that follows the rules — one axis, direct labels, a real legend, and a table view underneath.

visualization Visualization

Feb 8, 2026

Deliberately messy customers, in one paragraph

A thousand customer records broken in every way real data is broken: five date formats, inconsistent casing, padded whitespace, near-duplicate rows, mixed units, sentinel nulls, and encoding damage..

quality practice Data quality

Feb 7, 2026

Reader question — When not to build a model

The most valuable modelling skill is recognising the problems that do not need one, which is a large majority of the problems people bring you.

practice Machine learning

Feb 6, 2026

ride-hail-trips.borough: 5 values, 73.6% in the top three

5 values with 30.0% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Feb 5, 2026

Idempotent window replace, in short

Every write replaces a named slice rather than adding to a pile, so rerunning a job is safe and backfilling is routine.

pipelines Data engineering

Feb 4, 2026

The time coverage of city-air-quality

1,370 populated days across a 1,370-day span, and what that allows you to compute.

engineering quality Data engineering

Feb 3, 2026

units: mean 2, median 1

A 69.3% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

Feb 2, 2026

A running total that does not lie about ties

running total against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Feb 1, 2026

Is device_id + status the grain of sensor-telemetry?

26 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jan 31, 2026

2.1% of arr_delay_min is missing

167 blanks in one column, and the one-line check that decides whether dropping them is free.

quality statistics Statistics

Jan 30, 2026

What ordered_at actually contains in retail-orders

7,000 rows, 4,159 distinct values, and the one fact about ordered_at that changes how you query it.

quality practice Data quality

Jan 29, 2026

The arithmetic behind a cheap query

Column pruning times partition pruning on a real schema, and the function call that quietly defeats both.

performance warehousing Data platform