Blog, page 9

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Jan 28, 2026

The top of temp_c is 1.1× the 99th percentile

A max 1.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Jan 27, 2026

The overall average hides 30 different numbers

Real group means for one column across 30 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jan 26, 2026

r = 0.80 between o3_ppb and temp_c

A measured correlation of 0.80 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jan 25, 2026

Correction: Slowly changing dimension, type 2

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Analytics engineering

Jan 24, 2026

Charting csat by channel

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Jan 23, 2026

A five-minute use for Regression Lab

Fit an ordinary least squares line to any pair of columns and get the diagnostics that decide whether the fit means anything: residual plot, R², standard errors, and leverage.

visualization Visualization

Jan 22, 2026

What is deliberately wrong with retail-orders

Seven thousand e-commerce order lines across four channels and six categories, with a real Q4 seasonality bump, a returns flag, and discounting that varies by channel..

quality practice Data quality

Jan 21, 2026

Cutting a lesson down — Owning something that runs without you

The last module, and the one about what you owe other people — documentation, runbooks, handover, and the decision to turn something off.

practice Data platform

Jan 20, 2026

reading_at runs 2023-01-01 to 2023-02-28

59 populated days across a 59-day span, and what that allows you to compute.

engineering quality Data engineering

Jan 19, 2026

Pattern: Deterministic surrogate key

Derive the key from the content instead of an auto-increment, so reprocessing the same row produces the same key.

pipelines modelling Data engineering

Jan 18, 2026

The category that eats the chart

2 values with 55.3% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Jan 17, 2026

revenue_usd: mean 111, median 60

A 84.0% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

Jan 16, 2026

Keeping one row per key in sensor-telemetry

deduplicate by rank against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Jan 15, 2026

Is ticket_id + queue the grain of support-tickets?

4,000 distinct combinations against 4,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jan 14, 2026

Six things we got wrong, corrected in place

A public list of the errors readers found in the course, and what changed.

practice quality Analytics engineering

Jan 13, 2026

Check the blanks before you drop them

1,444 blanks in one column, and the one-line check that decides whether dropping them is free.

quality statistics Statistics

Jan 12, 2026

What unit_price_usd actually contains in retail-orders

7,000 rows, 5,287 distinct values, and the one fact about unit_price_usd that changes how you query it.

quality practice Data quality

Jan 11, 2026

The arithmetic behind a cheap query

Column pruning times partition pruning on a real schema, and the function call that quietly defeats both.

performance warehousing Data platform

Jan 10, 2026

The top of arr_delay_min is 3.1× the 99th percentile

A max 3.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Jan 9, 2026

The overall average hides 5 different numbers

Real group means for one column across 5 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jan 8, 2026

A essentially none correlation, and what it is not

A measured correlation of -0.01 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jan 7, 2026

Correction: Deduplicate by rank

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Analytics engineering

Jan 6, 2026

Charting fare_usd by payment_type

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Jan 5, 2026

A five-minute use for Correlation Explorer

A diverging heatmap of every numeric pair in a dataset, with a click-through scatter so you can check whether the number is telling the truth.

visualization Visualization