Blog, page 40

Field notes — one a day since May 2023. Things we shipped, things we broke, and numbers we went and checked.

2026 2025 2024 2023

1,201 posts. , or browse by year and tag below.

Jan 16, 2024

A timezone bug that cost us three days

`datetime.utcnow()` returns the right value with the wrong type, and everything downstream is a coin flip.

engineering quality Data engineering

Jan 15, 2024

Reading solar_mw before trusting it in grid-energy-load

8,760 rows, 3,538 distinct values, and the one fact about solar_mw that changes how you query it.

quality practice Data quality

Jan 14, 2024

Pattern: Anti-join

Find rows with no match on the other side — and avoid the NOT IN trap that silently returns nothing.

sql Analytics engineering

Jan 13, 2024

Cutting a lesson down — The anatomy of a silent failure

A walk through a real class of incident — the kind where every job is green, every test passes, and the number has been wrong for six weeks.

practice Data quality

Jan 12, 2024

The overall average hides 14 different numbers

Real group means for one column across 14 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jan 11, 2024

Reading hour_at before trusting it in grid-energy-load

8,760 rows, all distinct, and the one fact about hour_at that changes how you query it.

quality practice Data quality

Jan 10, 2024

A strong correlation, and what it is not

A measured correlation of 1.00 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jan 9, 2024

Cutting a lesson down — A working setup that will not embarrass you

The minimum professional toolchain — version control, environments, notebooks used correctly, and the habits that make work reproducible six months later by someone who is not you.

practice Data engineering

Jan 8, 2024

What dropped_out actually contains in clinical-trial

900 rows, 2 distinct values, and the one fact about dropped_out that changes how you query it.

quality practice Data quality

Jan 7, 2024

Counting rows is not testing grain

16 distinct combinations against 7,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jan 6, 2024

mrr_usd by region: a 21% spread

Real group means for one column across 4 segments, and what the pooled average conceals.

statistics practice Analytics practice

Jan 5, 2024

r = 0.08 between population and life_expectancy

A measured correlation of 0.08 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Jan 4, 2024

Reading prior_therapy before trusting it in clinical-trial

900 rows, 2 distinct values, and the one fact about prior_therapy that changes how you query it.

quality practice Data quality

Jan 3, 2024

Course note — Reading a result without fooling yourself

The experiment finished. Here is the order to look at things in, and the specific ways a real-looking result turns out not to be.

practice Experimentation

Jan 2, 2024

Counting rows is not testing grain

8,000 distinct combinations against 8,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Jan 1, 2024

The overall average hides 6 different numbers

Real group means for one column across 6 segments, and what the pooled average conceals.

statistics practice Analytics practice

Dec 31, 2023

A essentially none correlation, and what it is not

A measured correlation of -0.00 between two columns, what its square says, and the diagnostic that costs one line.

statistics Statistics

Dec 30, 2023

Is line_no + category the grain of retail-orders?

24 distinct combinations against 7,000 rows, and what that answers about the table's key.

sql quality warehousing Analytics engineering

Dec 29, 2023

Reading temp_c before trusting it in city-air-quality

5,480 rows, 352 distinct values, and the one fact about temp_c that changes how you query it.

quality practice Data quality

Dec 28, 2023

Course note — Turning a request into a question

"Can you pull the numbers for last quarter" is not a question. Converting it into one is the most under-taught skill in the job, and the one that most separates a senior analyst from a fast one.

practice Analytics practice

Dec 27, 2023