Field note · June 12, 2023
Course note — Orchestration, and what "it runs every night" costs
The moment analysis becomes a scheduled job, a whole category of problems arrives: dependencies, retries, alerting, and someone's phone at 3am.
Revised Orchestration, and what "it runs every night" costs this week. The change was small and the reason was not.
The moment analysis becomes a scheduled job, a whole category of problems arrives: dependencies, retries, alerting, and someone's phone at 3am.
The first draft explained the mechanism and stopped. What was missing was the failure: the thing that happens when you get it wrong, described concretely enough that someone recognises it later. A lesson that only teaches the correct version leaves you unable to spot the incorrect one, which is the situation you will actually be in.
It sits in the Shipping it and keeping it alive module. The exercise is a thinking one rather than a query one — some of this material is about deciding what to compute, and that does not need a keyboard.
Ordering matters here more than in most courses. It follows Communicating a result to someone who will not read your notebook and leads into Monitoring data, not just jobs, and reading it out of sequence mostly works but costs you the setup.
Free means free, and it also means we can rewrite it whenever it is wrong. No edition, no errata PDF, nothing to repurchase. The whole course is 36 lessons and the fixes land the day we find them.