Blog, page 23

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Feb 26, 2025

Randomised trial results, in one paragraph

A synthetic two-arm trial: 900 subjects, baseline and week-12 scores, adverse events, and a real but modest treatment effect that is easy to overstate if you analyse it badly..

quality practice Data quality

Feb 25, 2025

Course note — Frame the problem, then beat a stupid baseline

Most model projects fail at framing, not at modelling. The second most common failure is not knowing what "good" would have been without a model at all.

practice Machine learning

Feb 24, 2025

What units actually contains in retail-orders

7,000 rows, 5 distinct values, and the one fact about units that changes how you query it.

quality practice Data quality

Feb 23, 2025

ride-hail-trips.payment_type: 3 values, 100.0% in the top three

3 values with 62.3% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Feb 22, 2025

Counting rows is not testing grain

6,000 distinct combinations against 6,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Feb 21, 2025

Five minutes with rpm in sensor-telemetry

8,000 rows, 533 distinct values, and the one fact about rpm that changes how you query it.

quality practice Data quality

Feb 20, 2025

Counting your tests, in short

Twenty segments at α = 0.05 gives you a one-in-two chance of a false finding. Correct for it, or at minimum say how many you ran.

statistics experiments Experimentation

Feb 19, 2025

Counting rows is not testing grain

6,355 distinct combinations against 7,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Feb 18, 2025

CUPED gave us back 40% of our sample size for ten lines of code

Variance reduction using pre-experiment data, explained without the paper.

experiments statistics Experimentation

Feb 17, 2025

What distance_km actually contains in ride-hail-trips

8,000 rows, 1,572 distinct values, and the one fact about distance_km that changes how you query it.

quality practice Data quality

Feb 16, 2025

Keeping one row per key in ab-test-checkout

deduplicate by rank against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Feb 15, 2025

Before you clip the outliers

A max 3.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Feb 14, 2025

What line_no actually contains in retail-orders

7,000 rows, 4 distinct values, and the one fact about line_no that changes how you query it.

quality practice Data quality

Feb 13, 2025

seats_now by industry: a 57% spread

Real group means for one column across 8 segments, and what the pooled average conceals.

statistics practice Analytics practice

Feb 12, 2025

A essentially none correlation, and what it is not

A measured correlation of -0.05 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Feb 11, 2025

Charting bmi by arm

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Feb 10, 2025

Five minutes with discount_pct in retail-orders

7,000 rows, 42 distinct values, and the one fact about discount_pct that changes how you query it.

quality practice Data quality

Feb 9, 2025

An exercise on grid-energy-load

A year of hourly electricity demand, renewable generation, temperature and wholesale price for one balancing area — including the hours when price goes negative..

quality practice Data quality

Feb 8, 2025

Cutting a lesson down — Schema drift, and the contract that prevents it

Your pipeline will break because someone upstream renamed a column on a Thursday and told nobody. Here is how to make that a caught error instead of a silent one.

practice Data engineering

Feb 7, 2025

support-tickets.channel: 4 values, 87.8% in the top three

4 values with 34.2% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Feb 6, 2025

Five minutes with returned in retail-orders

7,000 rows, 2 distinct values, and the one fact about returned that changes how you query it.

quality practice Data quality

Feb 5, 2025

age by sex: a 1% spread

Real group means for one column across 2 segments, and what the pooled average conceals.

statistics practice Analytics practice

Feb 4, 2025

Pattern: Forward-chaining split

Validate the way you will deploy — train on the past, predict the future, and leave the gap your real prediction lag imposes.

modelling Machine learning

Feb 3, 2025

Finding the gaps in support-tickets

gaps and islands against a real schema, and the clause in it that is doing the work.

sql Analytics engineering