Blog, page 4

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

May 28, 2026

How do I explain a confidence interval to a stakeholder?

Do not explain it. Use it.

statistics practice Statistics

May 27, 2026

Why the datasets are seeded

Every dataset on this site is generated at build time from a fixed seed. Rebuild the site and the CSVs come out byte-identical.

engineering quality Data platform

May 26, 2026

Charting humidity_pct by status

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

May 25, 2026

Zero is not a small number

2.2% of one column is exactly zero. Whether that is a measurement or an absence changes every aggregate built on it.

quality statistics Data quality

May 24, 2026

42.1%, plus or minus what?

A point estimate of 42.1%, the interval around it, and what a tighter one would cost.

statistics Statistics

May 23, 2026

SQL Playground works offline

Write real SELECT queries against any dataset here and get results instantly — joins, aggregates, GROUP BY, HAVING and ORDER BY, with plain errors for the things it does not support.

visualization Visualization

May 22, 2026

What is deliberately wrong with city-air-quality

Daily pollutant readings for four cities over nearly four years, with strong seasonality, weekday effects, wildfire episodes, and genuine sensor outages..

quality practice Data quality

May 21, 2026

Five minutes with distance_mi in flight-delays

8,000 rows, 2,581 distinct values, and the one fact about distance_mi that changes how you query it.

quality practice Data quality

May 20, 2026

Reader question — Turning a request into a question

"Can you pull the numbers for last quarter" is not a question. Converting it into one is the most under-taught skill in the job, and the one that most separates a senior analyst from a fast one.

practice Analytics practice

May 19, 2026

Pattern: Deduplicate by rank

Keep the best row per key with a window function instead of a self-join or a GROUP BY that loses columns.

sql Analytics engineering

May 18, 2026

hour_at runs 2023-01-01 to 2023-12-31

365 populated days across a 365-day span, and what that allows you to compute.

engineering quality Data engineering

May 17, 2026

dep_delay_min by origin: a 113% spread

Real group means for one column across 14 segments, and what the pooled average conceals.

statistics practice Analytics practice

May 16, 2026

Check the blanks before you drop them

149 blanks in one column, and the one-line check that decides whether dropping them is free.

quality statistics Statistics

May 15, 2026

device_id is not unique, and the join knows

666.7 rows per key on one table, and the totals that quietly inherit the multiplier.

sql quality Analytics engineering

May 14, 2026

ab-test-checkout.variant: 3 values, 100.0% in the top three

3 values with 34.8% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

May 13, 2026

A essentially none correlation, and what it is not

A measured correlation of 0.00 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

May 12, 2026

Somebody translated the course into Portuguese

What an open licence actually buys, and what we are doing about it.

practice Analytics practice

May 11, 2026

first_response_min: mean 155, median 83

A 86.3% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

May 10, 2026

Is reading_at + device_id the grain of sensor-telemetry?

8,000 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

May 9, 2026

Sessions out of raw events in retail-orders

sessionisation against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

May 8, 2026

Charting sched_dep_hour by origin

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

May 7, 2026

The arithmetic behind a cheap query

Column pruning times partition pruning on a real schema, and the function call that quietly defeats both.

performance warehousing Data platform

May 6, 2026

The top of rpm is 1.1× the 99th percentile

A max 1.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

May 5, 2026

Correction: Deterministic surrogate key

The page claimed more than it should have. What it says now, and where the pattern stops applying.

quality practice Data engineering