Blog, page 24

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Feb 2, 2025

Before you clip the outliers

A max 3.0× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Feb 1, 2025

What baseline_score actually contains in clinical-trial

900 rows, 384 distinct values, and the one fact about baseline_score that changes how you query it.

quality practice Data quality

Jan 31, 2025

A essentially none correlation, and what it is not

A measured correlation of -0.01 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jan 30, 2025

Charting population by country

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Jan 29, 2025

What returned actually contains in retail-orders

7,000 rows, 2 distinct values, and the one fact about returned that changes how you query it.

quality practice Data quality

Jan 28, 2025

What is deliberately wrong with movie-ratings

Nine thousand user ratings over 260 films, with genres, release years, and the long-tail popularity distribution that makes naive recommenders look better than they are..

quality practice Data quality

Jan 27, 2025

Reader question — Joins that do not fan out

The most common wrong number in the world is caused by a join that multiplied rows, and the person who wrote it had no way to notice.

practice Analytics engineering

Jan 26, 2025

What life_expectancy actually contains in world-indicators

720 rows, 591 distinct values, and the one fact about life_expectancy that changes how you query it.

quality practice Data quality

Jan 25, 2025

Pattern: Contract check at the boundary

Validate the shape of incoming data where it enters your system — fail loudly on structure, warn on distribution.

quality pipelines Data quality

Jan 24, 2025

Sessions out of raw events in support-tickets

sessionisation against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Jan 23, 2025

Reading carrier before trusting it in flight-delays

8,000 rows, 6 distinct values, and the one fact about carrier that changes how you query it.

quality practice Data quality

Jan 22, 2025

The top of age is 1.1× the 99th percentile

A max 1.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Jan 21, 2025

Most of our experiments could never have worked

We audited two years of tests. Over half were incapable of detecting the effect they were looking for.

experiments statistics Experimentation

Jan 20, 2025

The category that eats the chart

10 values with 15.2% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Jan 19, 2025

Five minutes with signed_up_on in saas-subscriptions

2,500 rows, 947 distinct values, and the one fact about signed_up_on that changes how you query it.

quality practice Data quality

Jan 18, 2025

Counting rows is not testing grain

4,000 distinct combinations against 4,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jan 17, 2025

The overall average hides 2 different numbers

Real group means for one column across 2 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jan 16, 2025

A weak correlation, and what it is not

A measured correlation of 0.38 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jan 15, 2025

Charting mrr_usd by plan

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Jan 14, 2025

What is deliberately wrong with world-indicators

A tidy panel of thirty countries over twenty-four years with GDP per capita, life expectancy, population, CO₂ and internet penetration — small enough to hold in your head, rich enough for real modelling..

quality practice Data quality

Jan 13, 2025