Field note · May 7, 2025
Pattern: As-of feature computation
Every feature must be computable from information available at prediction time. Write down each one's timestamp, and the leakage disappears.
The shape: For every feature, state when its value becomes known. Anything after the cutoff is leakage. — that is As-of feature computation, and it comes up more than it should.
Every feature must be computable from information available at prediction time. Write down each one's timestamp, and the leakage disappears.
What makes it a pattern rather than a tip is that the wrong version is the one you write naturally. It reads correctly, it runs, and it returns something. The failure is in the result, not in the execution — which means the only defence is recognising the shape before you are in it.
Two datasets on this site have the shape built in: saas-subscriptions and support-tickets. Both are small enough to run the broken version, see the number, then run the corrected one and see it change.
It lives under modelling because the model is not where the mistake is. The mistake is upstream, in what the training rows knew.
The long-form treatment is in the course (baselines-and-framing, features-and-interpretation); the pattern page is the version to read at 4pm with a query open.
If you have a better formulation of this one, the repository takes issues. Several entries there are sharper than what we started with because someone pushed back.