Field note · May 24, 2023

Reader question — Monitoring data, not just jobs

Job monitoring tells you the code ran. Data monitoring tells you the numbers are right. Teams that only have the first find out about the second from a stakeholder.

1 min read ·Data platform ·practice

Reader question on Monitoring data, not just jobs, and the answer belongs somewhere more findable than an email.

Job monitoring tells you the code ran. Data monitoring tells you the numbers are right. Teams that only have the first find out about the second from a stakeholder.

The question was, roughly, "when does this stop applying?" — which is the right question and the one lessons routinely fail to answer. Every technique has a range of validity, and stating it is what separates a lesson from a recipe.

It sits in the Shipping it and keeping it alive module. The exercise is a thinking one rather than a query one — some of this material is about deciding what to compute, and that does not need a keyboard.

Ordering matters here more than in most courses. It follows Orchestration, and what "it runs every night" costs and leads into Owning something that runs without you, and reading it out of sequence mostly works but costs you the setup.

Free means free, and it also means we can rewrite it whenever it is wrong. No edition, no errata PDF, nothing to repurchase. The whole course is 36 lessons and the fixes land the day we find them.