Field note · November 16, 2023
Reader question — Uncertainty, sampling, and how much to trust a number
Every number computed from a sample is wrong. The useful question is by how much, and the machinery for answering it is smaller than you think.
Reader question on Uncertainty, sampling, and how much to trust a number, and the answer belongs somewhere more findable than an email.
Every number computed from a sample is wrong. The useful question is by how much, and the machinery for answering it is smaller than you think.
The question was, roughly, "when does this stop applying?" — which is the right question and the one lessons routinely fail to answer. Every technique has a range of validity, and stating it is what separates a lesson from a recipe.
It sits in the Statistics you will actually use module. The exercise is a thinking one rather than a query one — some of this material is about deciding what to compute, and that does not need a keyboard.
Ordering matters here more than in most courses. It follows Describing a column without lying and leads into The five tests that cover most questions, and reading it out of sequence mostly works but costs you the setup.
Free means free, and it also means we can rewrite it whenever it is wrong. No edition, no errata PDF, nothing to repurchase. The whole course is 36 lessons and the fixes land the day we find them.