Blog, page 16

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Aug 13, 2025

What Experiment Calculator will not do

Sample size before you launch, and a confidence interval after you stop. Both directions, with the assumptions written on the screen rather than buried in a footnote.

visualization Visualization

Aug 12, 2025

The time coverage of data-job-postings

966 populated days across a 1,001-day span, and what that allows you to compute.

engineering quality Data engineering

Aug 11, 2025

An exercise on dirty-customers

A thousand customer records broken in every way real data is broken: five date formats, inconsistent casing, padded whitespace, near-duplicate rows, mixed units, sentinel nulls, and encoding damage..

quality practice Data quality

Aug 10, 2025

Course note — Schema drift, and the contract that prevents it

Your pipeline will break because someone upstream renamed a column on a Thursday and told nobody. Here is how to make that a caught error instead of a silent one.

practice Data engineering

Aug 9, 2025

The left join that became an inner join, in short

A predicate on the right table in WHERE silently discards the unmatched rows you wrote a left join to keep.

sql Analytics engineering

Aug 8, 2025

data-job-postings.city: 10 values, 41.8% in the top three

10 values with 15.2% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Aug 7, 2025

passengers: mean 2, median 1

A 77.2% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

Aug 6, 2025

The top of unit_price_usd is 1.7× the 99th percentile

A max 1.7× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Aug 5, 2025

A running total that does not lie about ties

running total against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Aug 4, 2025

Counting rows is not testing grain

3,779 distinct combinations against 4,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Aug 3, 2025

Five minutes with passengers in ride-hail-trips

8,000 rows, 6 distinct values, and the one fact about passengers that changes how you query it.

quality practice Data quality

Aug 2, 2025

discount_pct by category: a 16% spread

Real group means for one column across 6 segments, and what the pooled average conceals.

statistics practice Analytics practice

Aug 1, 2025

r = 0.04 between temp_c and price_eur_mwh

A measured correlation of 0.04 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jul 31, 2025

Charting pm25 by city

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

Jul 30, 2025

Checkout A/B test, in one paragraph

Six thousand users randomised across a control and two variants, with conversion, revenue, device, and a novelty effect that fades over the first week..

quality practice Data quality

Jul 29, 2025

The CLT does not always save you

A tool for watching sampling distributions, and the picture that changed how I teach it.

statistics Statistics

Jul 28, 2025

Data Explorer works offline

A fast table with per-column profiling: type inference, missing counts, distinct values, quantiles, and a sparkline distribution for every numeric column.

visualization Visualization

Jul 27, 2025

Cutting a lesson down — Joins that do not fan out

The most common wrong number in the world is caused by a join that multiplied rows, and the person who wrote it had no way to notice.

practice Analytics engineering

Jul 26, 2025

Cohort retention triangle, in short

Group by when they started, measure how many remain N periods later — and remember that the bottom-right corner is young, not good.

statistics Analytics practice

Jul 25, 2025

ab-test-checkout.device: 3 values, 100.0% in the top three

3 values with 54.7% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Jul 24, 2025

Comparing a period to the one before it

period comparison against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

Jul 23, 2025

Counting rows is not testing grain

9,000 distinct combinations against 9,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jul 22, 2025

The average unit price usd is not the typical one

A 48.7% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

Jul 21, 2025

What site actually contains in sensor-telemetry

8,000 rows, 3 distinct values, and the one fact about site that changes how you query it.

quality practice Data quality