Field note · November 28, 2023
Ten months in, what writing the course has taught us
Mostly that we did not understand things we have been doing for a decade.
We are about ten months into writing this and roughly a third done, which against our original six-week estimate is going well.
The consistent surprise: explaining something you do every day reveals that you do not understand it.
Specific examples, all mine:
I have written slowly changing dimensions for years. Writing the lesson, I could not initially explain why the fact table stores the version-specific surrogate key rather than the natural key. I knew the pattern. I did not know the reason, which is that it resolves point-in-time attribution at load time so every downstream query becomes a simple equality join. Once I could say that, the whole lesson reorganised itself around it.
The p-value lesson has been rewritten four times, and the statistics side describes the current version as "the least wrong one".
A chart got deleted from a draft because writing the caption revealed it was making a claim the data did not support. That has happened three more times since. The theory now is that the caption is a lie detector for the chart.
The other thing we learned is about scope. The first draft of the SQL module was going to cover everything. It was unreadable. The current version covers five query shapes that appear constantly and says explicitly what it is leaving out. Much better, and much harder to write — deciding what not to include took longer than writing what remained.
If you are considering teaching something you know well: it is the most efficient way to find the holes in your own understanding, and it is considerably more uncomfortable than it sounds.