Blog, page 38

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,201 posts. , or browse by year and tag below.

Mar 4, 2024

Course note — Uncertainty, sampling, and how much to trust a number

Every number computed from a sample is wrong. The useful question is by how much, and the machinery for answering it is smaller than you think.

practice Statistics

Mar 3, 2024

Is line_no + order_id the grain of retail-orders?

7,000 distinct combinations against 7,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Mar 2, 2024

What load_mw actually contains in grid-energy-load

8,760 rows, 7,724 distinct values, and the one fact about load_mw that changes how you query it.

quality practice Data quality

Mar 1, 2024

Sessionisation, in short

Turn a stream of timestamped events into sessions by flagging the gaps and cumulative-summing the flag.

sql Analytics engineering

Feb 29, 2024

Before you clip the outliers

A max 1.1× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Feb 28, 2024

Is device_id + reading_at the grain of sensor-telemetry?

8,000 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Feb 27, 2024

Five minutes with temp_c in grid-energy-load

8,760 rows, 395 distinct values, and the one fact about temp_c that changes how you query it.

quality practice Data quality

Feb 26, 2024

The overall average hides 3 different numbers

Real group means for one column across 3 segments, and what the pooled average conceals.

statistics practice Analytics practice

Feb 25, 2024

r = 0.09 between duration_min and surge_multiplier

A measured correlation of 0.09 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Feb 24, 2024

What vibration_mm_s actually contains in sensor-telemetry

8,000 rows, 2,369 distinct values, and the one fact about vibration_mm_s that changes how you query it.

quality practice Data quality

Feb 23, 2024

Reader question — Facts, dimensions, and picking a grain

Dimensional modelling is thirty years old and still the right default, because it optimises for the thing that actually matters — an analyst being able to answer a question without asking anyone.

practice Analytics engineering

Feb 22, 2024

Pattern: Sessionisation

Turn a stream of timestamped events into sessions by flagging the gaps and cumulative-summing the flag.

sql Analytics engineering

Feb 21, 2024

Before you clip the outliers

A max 1.7× the 99th percentile, and the three different things that could be.

statistics quality Statistics

Feb 20, 2024

We shipped a Simpson's paradox to the exec team

Every region improved. The company average got worse. Both were true.

statistics practice quality Statistics

Feb 19, 2024

Five minutes with vibration_mm_s in sensor-telemetry

8,000 rows, 2,369 distinct values, and the one fact about vibration_mm_s that changes how you query it.

quality practice Data quality

Feb 18, 2024

seats_now by region: a 20% spread

Real group means for one column across 4 segments, and what the pooled average conceals.

statistics practice Analytics practice

Feb 17, 2024

A essentially none correlation, and what it is not

A measured correlation of 0.11 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Feb 16, 2024

What variant actually contains in ab-test-checkout

6,000 rows, 3 distinct values, and the one fact about variant that changes how you query it.

quality practice Data quality

Feb 15, 2024

Course note — Describing a column without lying

The mean is the default summary and it is frequently the wrong one. Knowing when — and what to reach for instead — is most of applied statistics in this job.

practice Statistics

Feb 14, 2024