Blog, page 43

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,245 posts. , or browse by year and tag below.

Dec 19, 2023

Year one

What exists, what does not, and what we got wrong.

practice Analytics practice

Dec 18, 2023

Reader question — Frame the problem, then beat a stupid baseline

Most model projects fail at framing, not at modelling. The second most common failure is not knowing what "good" would have been without a model at all.

practice Machine learning

Dec 17, 2023

Reading queue before trusting it in support-tickets

4,000 rows, 5 distinct values, and the one fact about queue that changes how you query it.

quality practice Data quality

Dec 16, 2023

Is movie_id + genre the grain of movie-ratings?

260 distinct combinations against 9,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Dec 15, 2023

The overall average hides 6 different numbers

Real group means for one column across 6 segments, and what the pooled average conceals.

statistics practice Analytics practice

Dec 14, 2023

Five minutes with distance_km in ride-hail-trips

8,000 rows, 1,572 distinct values, and the one fact about distance_km that changes how you query it.

quality practice Data quality

Dec 13, 2023

r = 0.45 between pm10 and no2_ppb

A measured correlation of 0.45 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Dec 12, 2023

Course note — Validation that matches deployment

Cross-validation is not one technique. Choosing the wrong variant is how models get shipped with an offline score that has no relationship to what happens next.

practice Machine learning

Dec 11, 2023

Reading exposed_on before trusting it in ab-test-checkout

6,000 rows, 28 distinct values, and the one fact about exposed_on that changes how you query it.

quality practice Data quality

Dec 10, 2023

Is resolved_at + queue the grain of support-tickets?

3,780 distinct combinations against 4,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Dec 9, 2023

r = 0.98 between seats and mrr_usd

A measured correlation of 0.98 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Dec 8, 2023

Reading no2_ppb before trusting it in city-air-quality

5,480 rows, 450 distinct values, and the one fact about no2_ppb that changes how you query it.

quality practice Data quality

Dec 7, 2023

Counting rows is not testing grain

4,169 distinct combinations against 7,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Dec 6, 2023

sessions by device: a 3% spread

Real group means for one column across 3 segments, and what the pooled average conceals.

statistics practice Analytics practice

Dec 5, 2023

Reading signed_up_on before trusting it in saas-subscriptions

2,500 rows, 947 distinct values, and the one fact about signed_up_on that changes how you query it.

quality practice Data quality

Dec 4, 2023

Reader question — Owning something that runs without you

The last module, and the one about what you owe other people — documentation, runbooks, handover, and the decision to turn something off.

practice Data platform

Dec 3, 2023

The overall average hides 6 different numbers

Real group means for one column across 6 segments, and what the pooled average conceals.

statistics practice Analytics practice

Dec 2, 2023

What reading_at actually contains in sensor-telemetry

8,000 rows, all distinct, and the one fact about reading_at that changes how you query it.

quality practice Data quality

Dec 1, 2023

A moderate correlation, and what it is not

A measured correlation of 0.53 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Nov 30, 2023

Cutting a lesson down — What a pipeline is really doing

Extract, load, transform — why the industry flipped the last two letters, and what a pipeline owes you beyond moving bytes.

practice Data engineering

Nov 29, 2023

What site actually contains in clinical-trial

900 rows, 6 distinct values, and the one fact about site that changes how you query it.

quality practice Data quality

Nov 28, 2023

Ten months in, what writing the course has taught us

Mostly that we did not understand things we have been doing for a decade.

practice Analytics engineering

Nov 27, 2023

Is user_id + variant the grain of ab-test-checkout?

6,000 distinct combinations against 6,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Nov 26, 2023

The overall average hides 4 different numbers

Real group means for one column across 4 segments, and what the pooled average conceals.

statistics practice Analytics practice