Blog, page 25

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Jan 9, 2025

Pattern: Gaps and islands

Group consecutive runs — of dates, of statuses, of anything — by subtracting a row number from the value.

sql Analytics engineering

Jan 8, 2025

flight-delays.origin: 14 values, 22.6% in the top three

14 values with 7.6% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Jan 7, 2025

What channel actually contains in support-tickets

4,000 rows, 4 distinct values, and the one fact about channel that changes how you query it.

quality practice Data quality

Jan 6, 2025

Counting rows is not testing grain

8,000 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jan 5, 2025

Before you clip the outliers

A max 2.7× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Jan 4, 2025

What is_returning actually contains in ab-test-checkout

6,000 rows, 2 distinct values, and the one fact about is_returning that changes how you query it.

quality practice Data quality

Jan 3, 2025

salary_min_usd by country: a 379% spread

Real group means for one column across 7 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jan 2, 2025

A moderate correlation, and what it is not

A measured correlation of 0.51 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jan 1, 2025

Charting vibration_mm_s by status

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Dec 31, 2024

An exercise on ab-test-checkout

Six thousand users randomised across a control and two variants, with conversion, revenue, device, and a novelty effect that fades over the first week..

quality practice Data quality

Dec 30, 2024

Course note — One metric, one definition

The reason three dashboards show three different revenue numbers is never a bug. It is three people who each wrote their own definition, and all of them were reasonable.

practice Analytics engineering

Dec 29, 2024

Five minutes with flight_date in flight-delays

8,000 rows, 365 distinct values, and the one fact about flight_date that changes how you query it.

quality practice Data quality

Dec 28, 2024

Difference-in-differences, in short

When a change hits one group and not another, compare the change over time rather than the levels. Any time-invariant difference cancels out.

statistics experiments Experimentation

Dec 27, 2024

The category that eats the chart

8 values with 13.4% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Dec 26, 2024

Five minutes with adverse_event in clinical-trial

900 rows, 4 distinct values, and the one fact about adverse_event that changes how you query it.

quality practice Data quality

Dec 25, 2024

Finding the gaps in ride-hail-trips

gaps and islands against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Dec 24, 2024

Counting rows is not testing grain

8,000 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Dec 23, 2024

The top of solar_mw is 1.2× the 99th percentile

A max 1.2× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Dec 22, 2024

The overall average hides 2 different numbers

Real group means for one column across 2 segments, and what the pooled average conceals.

statistics practice Analytics practice

Dec 21, 2024

Five minutes with o3_ppb in city-air-quality

5,480 rows, 522 distinct values, and the one fact about o3_ppb that changes how you query it.

quality practice Data quality

Dec 20, 2024

r = 0.70 between fare_usd and tip_usd

A measured correlation of 0.70 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Dec 19, 2024

Charting dep_delay_min by origin

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Dec 18, 2024

Reading bmi before trusting it in clinical-trial

900 rows, 199 distinct values, and the one fact about bmi that changes how you query it.

quality practice Data quality

Dec 17, 2024

Year two

Six modules, four tools, fourteen datasets, and the thing we keep getting wrong.

practice Analytics practice