Blog, page 34

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,201 posts. , or browse by year and tag below.

Jun 8, 2024

What release_year actually contains in movie-ratings

9,000 rows, 53 distinct values, and the one fact about release_year that changes how you query it.

quality practice Data quality

Jun 7, 2024

data-job-postings.remote: 3 values, 100.0% in the top three

3 values with 45.5% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

Jun 6, 2024

Counting rows is not testing grain

8,000 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jun 5, 2024

Reading week12_score before trusting it in clinical-trial

900 rows, 434 distinct values, and the one fact about week12_score that changes how you query it.

quality practice Data quality

Jun 4, 2024

sessions by variant: a 2% spread

Real group means for one column across 3 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jun 3, 2024

r = 0.00 between flight_no and sched_dep_hour

A measured correlation of 0.00 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jun 2, 2024

What country actually contains in retail-orders

7,000 rows, 8 distinct values, and the one fact about country that changes how you query it.

quality practice Data quality

Jun 1, 2024

Charting salary_max_usd by city

Two facts about the columns decide the chart. Cardinality and skew, in that order.

visualization Visualization

May 31, 2024

Reader question — Slowly changing dimensions, and the "as of when" problem

A customer moved from Berlin to Madrid last March. Was last February's revenue German or Spanish? The answer is a design decision, and pretending otherwise is how reports stop reconciling.

practice Analytics engineering

May 30, 2024

Five minutes with tip_usd in ride-hail-trips

8,000 rows, 1,276 distinct values, and the one fact about tip_usd that changes how you query it.

quality practice Data quality

May 29, 2024

Pattern: Bootstrap any statistic

When you cannot remember the standard error formula — or there is not one — resample. It works for medians, percentiles and ratios alike.

statistics Statistics

May 28, 2024

The category that eats the chart

8 values with 37.9% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization

May 27, 2024

Reading population before trusting it in world-indicators

720 rows, all distinct, and the one fact about population that changes how you query it.

quality practice Data quality

May 26, 2024

Counting rows is not testing grain

4,000 distinct combinations against 4,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

May 25, 2024

The top of week12_score is 1.2× the 99th percentile

A max 1.2× the 99th percentile, and the three different things that could be.

statistics quality Statistics

May 24, 2024

What channel actually contains in saas-subscriptions

2,500 rows, 6 distinct values, and the one fact about channel that changes how you query it.

quality practice Data quality

May 23, 2024

discount_pct by channel: a 280% spread

Real group means for one column across 4 segments, and what the pooled average conceals.

statistics practice Analytics practice

May 22, 2024

A moderate correlation, and what it is not

A measured correlation of 0.45 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

May 21, 2024

We were wrong about streaming

Two years of Kafka for a use case that a five-minute batch would have served.

engineering pipelines Data engineering

May 20, 2024

What price_eur_mwh actually contains in grid-energy-load

8,760 rows, 4,568 distinct values, and the one fact about price_eur_mwh that changes how you query it.

quality practice Data quality

May 19, 2024

Reader question — Colour is an encoding, not decoration

Colour has exactly four jobs in a chart. Using it for anything else adds noise, and using it carelessly excludes about one in twelve of your readers.

practice Visualization

May 18, 2024

Conditional pivot, in short

Reshape long to wide with CASE inside an aggregate. Portable, readable, and it works everywhere.

sql Analytics engineering

May 17, 2024

Reading dep_delay_min before trusting it in flight-delays

8,000 rows, 251 distinct values, and the one fact about dep_delay_min that changes how you query it.

quality practice Data quality

May 16, 2024

support-tickets.queue: 5 values, 72.2% in the top three

5 values with 32.1% concentrated in one of them, and what that does to every chart downstream.

visualization practice Visualization