Blog, page 47

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Jul 31, 2023

What internet_pct actually contains in world-indicators

720 rows, 401 distinct values, and the one fact about internet_pct that changes how you query it.

quality practice Data quality

Jul 30, 2023

Reader question — Validation that matches deployment

Cross-validation is not one technique. Choosing the wrong variant is how models get shipped with an offline score that has no relationship to what happens next.

practice Machine learning

Jul 29, 2023

r = 0.08 between temp_c and humidity_pct

A measured correlation of 0.08 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jul 28, 2023

Five minutes with unit_price_usd in retail-orders

7,000 rows, 5,287 distinct values, and the one fact about unit_price_usd that changes how you query it.

quality practice Data quality

Jul 27, 2023

Is account_id + region the grain of saas-subscriptions?

2,500 distinct combinations against 2,500 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jul 26, 2023

Cutting a lesson down — Uncertainty, sampling, and how much to trust a number

Every number computed from a sample is wrong. The useful question is by how much, and the machinery for answering it is smaller than you think.

practice Statistics

Jul 25, 2023

We deleted 34 dashboards and nobody complained

An audit, a two-week deprecation window, and a surprising amount of reclaimed attention.

visualization practice Visualization

Jul 24, 2023

r = 0.26 between year and life_expectancy

A measured correlation of 0.26 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jul 23, 2023

Reading revenue_usd before trusting it in retail-orders

7,000 rows, 5,719 distinct values, and the one fact about revenue_usd that changes how you query it.

quality practice Data quality

Jul 22, 2023

vibration_mm_s by site: a 29% spread

Real group means for one column across 3 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jul 21, 2023

A essentially none correlation, and what it is not

A measured correlation of 0.01 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jul 20, 2023

Reading trip_id before trusting it in ride-hail-trips

8,000 rows, all distinct, and the one fact about trip_id that changes how you query it.

quality practice Data quality

Jul 19, 2023

The overall average hides 6 different numbers

Real group means for one column across 6 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jul 18, 2023

A strong correlation, and what it is not

A measured correlation of 0.71 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jul 17, 2023

Reading status before trusting it in sensor-telemetry

8,000 rows, 3 distinct values, and the one fact about status that changes how you query it.

quality practice Data quality

Jul 16, 2023

The overall average hides 30 different numbers

Real group means for one column across 30 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jul 15, 2023

A strong correlation, and what it is not

A measured correlation of 0.96 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jul 14, 2023

Reading revenue_usd before trusting it in ab-test-checkout

6,000 rows, 603 distinct values, and the one fact about revenue_usd that changes how you query it.

quality practice Data quality

Jul 13, 2023

The overall average hides 10 different numbers

Real group means for one column across 10 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jul 12, 2023

A essentially none correlation, and what it is not

A measured correlation of -0.00 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jul 11, 2023

What year actually contains in world-indicators

720 rows, 24 distinct values, and the one fact about year that changes how you query it.

quality practice Data quality

Jul 10, 2023

Counting rows is not testing grain

6,000 distinct combinations against 6,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jul 9, 2023

Course note — Batch, streaming, and the honest difference

Streaming is not "batch but faster". It is a different consistency model with different failure modes, and most teams that adopt it did not need to.

practice Data engineering

Jul 8, 2023

The overall average hides 3 different numbers

Real group means for one column across 3 segments, and what the pooled average conceals.

statistics practice Analytics practice