Field note · June 10, 2024
Baseline before model, in short
Build the dumbest thing that could work and measure it properly. Your model must beat it on the same split, or it has done nothing.
Added a note to Baseline before model today, which is a good excuse to say the short version here.
Build the dumbest thing that could work and measure it properly. Your model must beat it on the same split, or it has done nothing.
What makes it a pattern rather than a tip is that the wrong version is the one you write naturally. It reads correctly, it runs, and it returns something. The failure is in the result, not in the execution — which means the only defence is recognising the shape before you are in it.
Two datasets on this site have the shape built in: grid-energy-load and support-tickets. Both are small enough to run the broken version, see the number, then run the corrected one and see it change.
It lives under modelling because the model is not where the mistake is. The mistake is upstream, in what the training rows knew.
The long-form treatment is in the course (baselines-and-framing, when-not-to-model); the pattern page is the version to read at 4pm with a query open.
The pattern page has the version that holds and the version that looks right and is not, side by side. Reading them together is the point; either one alone is just code.