Blog, page 46

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Aug 24, 2023

Cutting a lesson down — Validation that matches deployment

Cross-validation is not one technique. Choosing the wrong variant is how models get shipped with an offline score that has no relationship to what happens next.

practice Machine learning

Aug 23, 2023

What gdp_per_capita_usd actually contains in world-indicators

720 rows, 719 distinct values, and the one fact about gdp_per_capita_usd that changes how you query it.

quality practice Data quality

Aug 22, 2023

We p-hacked ourselves and caught it in review

Fourteen segments, one significant result, and the correction that made it go away.

statistics experiments Experimentation

Aug 21, 2023

r = -0.02 between sessions and revenue_usd

A measured correlation of -0.02 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Aug 20, 2023

Cutting a lesson down — Colour is an encoding, not decoration

Colour has exactly four jobs in a chart. Using it for anything else adds noise, and using it carelessly excludes about one in twelve of your readers.

practice Visualization

Aug 19, 2023

duration_min by borough: a 254% spread

Real group means for one column across 5 segments, and what the pooled average conceals.

statistics practice Analytics practice

Aug 18, 2023

A essentially none correlation, and what it is not

A measured correlation of -0.05 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Aug 17, 2023

Reading o3_ppb before trusting it in city-air-quality

5,480 rows, 522 distinct values, and the one fact about o3_ppb that changes how you query it.

quality practice Data quality

Aug 16, 2023

unit_price_usd by channel: a 6% spread

Real group means for one column across 4 segments, and what the pooled average conceals.

statistics practice Analytics practice

Aug 15, 2023

r = 0.01 between arr_delay_min and distance_mi

A measured correlation of 0.01 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Aug 14, 2023

Five minutes with site in sensor-telemetry

8,000 rows, 3 distinct values, and the one fact about site that changes how you query it.

quality practice Data quality

Aug 13, 2023

distance_km by borough: a 331% spread

Real group means for one column across 5 segments, and what the pooled average conceals.

statistics practice Analytics practice

Aug 12, 2023

Cutting a lesson down — Window functions, properly

The single highest-leverage thing to learn in SQL. Running totals, rankings, period-over-period, deduplication, sessionisation — all of it collapses into one construct.

practice Analytics engineering

Aug 11, 2023

A essentially none correlation, and what it is not

A measured correlation of 0.10 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Aug 10, 2023

temp_c by site: a 3% spread

Real group means for one column across 3 segments, and what the pooled average conceals.

statistics practice Analytics practice

Aug 9, 2023

Five minutes with surge_multiplier in ride-hail-trips

8,000 rows, 24 distinct values, and the one fact about surge_multiplier that changes how you query it.

quality practice Data quality

Aug 8, 2023

A essentially none correlation, and what it is not

A measured correlation of 0.01 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Aug 7, 2023

year by country: a wide spread

Real group means for one column across 30 segments, and what the pooled average conceals.

statistics practice Analytics practice

Aug 6, 2023

Five minutes with load_mw in grid-energy-load

8,760 rows, 7,724 distinct values, and the one fact about load_mw that changes how you query it.

quality practice Data quality

Aug 5, 2023

A essentially none correlation, and what it is not

A measured correlation of 0.00 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Aug 4, 2023

The overall average hides 3 different numbers

Real group means for one column across 3 segments, and what the pooled average conceals.

statistics practice Analytics practice

Aug 3, 2023

Five minutes with arr_delay_min in flight-delays

8,000 rows, 261 distinct values, and the one fact about arr_delay_min that changes how you query it.

quality practice Data quality

Aug 2, 2023

A moderate correlation, and what it is not

A measured correlation of 0.62 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Aug 1, 2023

revenue_usd by category: a 1,000% spread

Real group means for one column across 6 segments, and what the pooled average conceals.

statistics practice Analytics practice