Blog, page 12

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Nov 17, 2025

A weak correlation, and what it is not

A measured correlation of 0.22 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Nov 16, 2025

Correction: Difference-in-differences

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Experimentation

Nov 15, 2025

Charting release_year by genre

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Nov 14, 2025

Distribution Lab works offline

Sample from seven distributions, watch the central limit theorem happen at whatever sample size you choose, and see exactly when the mean stops being a useful summary.

visualization Visualization

Nov 13, 2025

An exercise on flight-delays

Eight thousand domestic flights across six carriers and fourteen airports, with departure and arrival delay, cancellations, and a delay distribution that is long-tailed rather than normal..

quality practice Data quality

Nov 12, 2025

Course note — Window functions, properly

The single highest-leverage thing to learn in SQL. Running totals, rankings, period-over-period, deduplication, sessionisation — all of it collapses into one construct.

practice Analytics engineering

Nov 11, 2025

Is order_id + line_no the grain of retail-orders?

7,000 distinct combinations against 7,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Nov 10, 2025

Conditional pivot, in short

Reshape long to wide with CASE inside an aggregate. Portable, readable, and it works everywhere.

sql Analytics engineering

Nov 9, 2025

The time coverage of support-tickets

400 populated days across a 400-day span, and what that allows you to compute.

engineering quality Data engineering

Nov 8, 2025

The category that eats the chart

3 values with 62.3% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Nov 7, 2025

The average units is not the typical one

A 69.3% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

Nov 6, 2025

Sessions out of raw events in ab-test-checkout

sessionisation against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Nov 5, 2025

Five minutes with subject_id in clinical-trial

900 rows, all distinct, and the one fact about subject_id that changes how you query it.

quality practice Data quality

Nov 4, 2025

Reading 3 columns instead of 10

Column pruning times partition pruning on a real schema, and the function call that quietly defeats both.

performance warehousing Data platform

Nov 3, 2025

The top of discount_pct is 1.1× the 99th percentile

A max 1.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Nov 2, 2025

seats by plan: a 21,443% spread

Real group means for one column across 4 segments, and what the pooled average conceals.

statistics practice Analytics practice

Nov 1, 2025

A essentially none correlation, and what it is not

A measured correlation of 0.08 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Oct 31, 2025

Correction: Conditional pivot

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Analytics engineering

Oct 30, 2025

Charting first_response_min by queue

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Oct 29, 2025

The chart rules behind Correlation Explorer

A diverging heatmap of every numeric pair in a dataset, with a click-through scatter so you can check whether the number is telling the truth.

visualization Visualization

Oct 28, 2025

The question that kills the most projects

If this were 100% accurate, what would change, and what is that worth?

practice modelling Analytics practice

Oct 27, 2025

The category that eats the chart

5 values with 30.0% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Oct 26, 2025

An exercise on data-job-postings

Thirty-five hundred data job ads with titles, seniority, location, remote policy, salary bands, and a pipe-delimited skills column that needs parsing before it is useful..

quality practice Data quality

Oct 25, 2025

Cutting a lesson down — Idempotency, backfills, and yesterday arriving twice

The single property that separates a pipeline you can operate from one you have to babysit — and the operation that will test it every time.

practice Data engineering