Blog, page 26

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Dec 16, 2024

Cutting a lesson down — Facts, dimensions, and picking a grain

Dimensional modelling is thirty years old and still the right default, because it optimises for the thing that actually matters — an analyst being able to answer a question without asking anyone.

practice Analytics engineering

Dec 15, 2024

Forward-chaining split, in short

Validate the way you will deploy — train on the past, predict the future, and leave the gap your real prediction lag imposes.

modelling Machine learning

Dec 14, 2024

Reading category before trusting it in retail-orders

7,000 rows, 6 distinct values, and the one fact about category that changes how you query it.

quality practice Data quality

Dec 13, 2024

The overall average hides 5 different numbers

Real group means for one column across 5 segments, and what the pooled average conceals.

statistics practice Analytics practice

Dec 12, 2024

movie-ratings.genre: 8 values, 54.8% in the top three

8 values with 21.7% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Dec 11, 2024

Five minutes with churned_at in saas-subscriptions

2,500 rows, 624 distinct values, and the one fact about churned_at that changes how you query it.

quality practice Data quality

Dec 10, 2024

A running total that does not lie about ties

running total against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Dec 9, 2024

A strong correlation, and what it is not

A measured correlation of 0.77 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Dec 8, 2024

Reading line_no before trusting it in retail-orders

7,000 rows, 4 distinct values, and the one fact about line_no that changes how you query it.

quality practice Data quality

Dec 7, 2024

Counting rows is not testing grain

4,000 distinct combinations against 4,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Dec 6, 2024

The top of temp_c is 1.4× the 99th percentile

A max 1.4× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Dec 5, 2024

Reading surge_multiplier before trusting it in ride-hail-trips

8,000 rows, 24 distinct values, and the one fact about surge_multiplier that changes how you query it.

quality practice Data quality

Dec 4, 2024

Charting gdp_per_capita_usd by region

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Dec 3, 2024

Cutting a lesson down — Features, and explaining what the model did

Feature engineering is where domain knowledge enters a model, and interpretation is where it leaves. Both are more valuable than algorithm selection.

practice Machine learning

Dec 2, 2024

Five minutes with sex in clinical-trial

900 rows, 2 distinct values, and the one fact about sex that changes how you query it.

quality practice Data quality

Dec 1, 2024

Median beside the mean, in short

Report both. The gap between them is a free skew diagnostic, and it tells the reader which number to trust.

statistics Statistics

Nov 30, 2024

The category that eats the chart

14 values with 7.5% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Nov 29, 2024

Reading distance_km before trusting it in ride-hail-trips

8,000 rows, 1,572 distinct values, and the one fact about distance_km that changes how you query it.

quality practice Data quality

Nov 28, 2024

Comparing a period to the one before it

period comparison against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Nov 27, 2024

Before you clip the outliers

A max 1.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Nov 26, 2024

Three dashboards, three revenue numbers, ninety minutes

Nobody's SQL was wrong. That was the problem.

warehousing practice Analytics engineering

Nov 25, 2024

What user_id actually contains in ab-test-checkout

6,000 rows, all distinct, and the one fact about user_id that changes how you query it.

quality practice Data quality

Nov 24, 2024

Counting rows is not testing grain

7,663 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Nov 23, 2024

mrr_usd by plan: a 13,612% spread

Real group means for one column across 4 segments, and what the pooled average conceals.

statistics practice Analytics practice