Blog, page 19

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,200 posts. , or browse by year and tag below.

Jun 2, 2025

Pattern: Date spine

Join to a generated calendar so days with no activity appear as zero instead of vanishing.

sql Analytics engineering

Jun 1, 2025

world-indicators.region: 5 values, 70.0% in the top three

5 values with 26.7% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

May 31, 2025

Sessions out of raw events in sensor-telemetry

sessionisation against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

May 30, 2025

sched_dep_hour by origin: a 4% spread

Real group means for one column across 14 segments, and what the pooled average conceals.

statistics practice Analytics practice

May 29, 2025

Five minutes with opened_at in support-tickets

4,000 rows, 3,990 distinct values, and the one fact about opened_at that changes how you query it.

quality practice Data quality

May 28, 2025

line_no: mean 2, median 1

A 58.5% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

May 27, 2025

Our most popular dataset is the broken one

Fourteen datasets, and the one people actually use is deliberately full of errors.

quality Data quality

May 26, 2025

Before you clip the outliers

A max 3.2× the 99th percentile, and the three different things that could be.

statistics quality Statistics

May 25, 2025

A essentially none correlation, and what it is not

A measured correlation of 0.01 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

May 24, 2025

Charting first_response_min by channel

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

May 23, 2025

What Distribution Lab will not do

Sample from seven distributions, watch the central limit theorem happen at whatever sample size you choose, and see exactly when the mean stops being a useful summary.

visualization Visualization

May 22, 2025

What is deliberately wrong with dirty-customers

A thousand customer records broken in every way real data is broken: five date formats, inconsistent casing, padded whitespace, near-duplicate rows, mixed units, sentinel nulls, and encoding damage..

quality practice Data quality

May 21, 2025

Course note — Testing data like you test code

Your transformation logic is tested by unit tests. The data flowing through it is not, and that is where the failures actually happen.

practice Data quality

May 20, 2025

Pattern: Counting your tests

Twenty segments at α = 0.05 gives you a one-in-two chance of a false finding. Correct for it, or at minimum say how many you ran.

statistics experiments Experimentation

May 19, 2025

The category that eats the chart

3 values with 35.3% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

May 18, 2025

A running total that does not lie about ties

running total against a real schema, and the clause in it that is doing the work.

sql Analytics engineering

May 17, 2025

Counting rows is not testing grain

900 distinct combinations against 900 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

May 16, 2025

unit_price_usd: mean 72.66, median 48.86

A 48.7% gap between mean and median on one column, and which of the two answers the question you were asked.

statistics practice Statistics

May 15, 2025

The top of temp_c is 1.1× the 99th percentile

A max 1.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

May 14, 2025

What humidity_pct actually contains in sensor-telemetry

8,000 rows, 260 distinct values, and the one fact about humidity_pct that changes how you query it.

quality practice Data quality

May 13, 2025

The overall average hides 7 different numbers

Real group means for one column across 7 segments, and what the pooled average conceals.

statistics practice Analytics practice

May 12, 2025

r = -0.01 between distance_km and surge_multiplier

A measured correlation of -0.01 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

May 11, 2025

Charting pm10 by city

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

May 10, 2025

The chart rules behind Chart Builder

Point it at any dataset, choose an x, a y, and a grouping, and get a chart that follows the rules — one axis, direct labels, a real legend, and a table view underneath.

visualization Visualization