Field note · January 14, 2026

Six things we got wrong, corrected in place

A public list of the errors readers found in the course, and what changed.

1 min read ·Analytics engineering ·practice quality

We said corrections would go in the page with a note rather than being silently edited. Here is the first year's worth, collected.

1. Cohort retention off by a month. The retention SQL used <= on a truncated churn month. An account that churned on 14 March was active in March, so < is right. One character, and it shifted month-1 retention by several points. Found by a reader within a week.

2. not in with nulls. The joins lesson described not in against a subquery as merely "risky". It is worse than that: with a single null in the subquery it returns zero rows, always, silently. Rewritten to say so plainly and recommend not exists.

3. Welch versus Student. The original tests lesson mentioned Welch's t-test as an option for unequal variances. That understates it — Welch should essentially always be the default, since it costs almost nothing when variances are equal and is substantially more correct when they are not.

4. The 1.5×IQR outlier rule. Presented without qualification. On lognormal data it flags a large fraction of perfectly legitimate rows, because the distribution genuinely has a long tail. Now framed as a convention that assumes symmetry.

5. Leverage threshold in the regression tool. We used the textbook 3p/n rule, which at n = 8,000 rings several percent of the cloud in orange and stops meaning anything. Now rings the worst twenty points, which is what the marker is actually for.

6. Parquet compression default. The file formats article recommended snappy. zstd is smaller at comparable speed and is a better default on modern engines. Updated.

Three of these came from readers, which is three more than we would have found ourselves. The issue tracker is open and we would rather be corrected than right.