Forward-chaining split

You meet it when#

Anything with a time dimension, which is most business prediction problems. A random k-fold split lets the model train on Thursday to predict Wednesday. In production it will only ever have the past.

The pattern#

from sklearn.model_selection import TimeSeriesSplit

splitter = TimeSeriesSplit(n_splits=5, test_size=24 * 7, gap=24)
for train_idx, test_idx in splitter.split(X):
    ...

Two details matter more than the technique:

The gap. If the model predicts 24 hours ahead, leave 24 hours between train and test. Without it you validate on a horizon you will never have.

Retrain per fold, the way production does. Validating one model against five time slices, when production retrains weekly, measures the wrong thing.

The other two splits that are not random#

Grouped. Multiple rows per user, device or account. If the same group appears on both sides, the metric is inflated and you find out in production.

from sklearn.model_selection import StratifiedGroupKFold
for train_idx, test_idx in StratifiedGroupKFold(5).split(X, y, groups=user_ids):
    ...

Distribution-shifted. If you expect to deploy into a new market or season, hold out a whole geography or the most recent period rather than a random slice. It will score worse. It will be honest.

Preprocessing goes inside the pipeline#

Fitting a scaler or imputer before splitting leaks test-set statistics into training. Subtle, universal, and completely avoidable:

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression

pipeline = Pipeline([
    ("impute", SimpleImputer(strategy="median")),   # median from the TRAIN fold only
    ("scale",  StandardScaler()),
    ("model",  LogisticRegression(max_iter=1000)),
])

Report the spread, not the point#

ROC AUC: 0.741 ± 0.028   (folds: 0.712, 0.735, 0.744, 0.757, 0.757)

If model A is 0.741 ± 0.028 and model B is 0.748 ± 0.031, they are indistinguishable. Ship the simpler, faster, more maintainable one. An enormous amount of modelling effort goes into differences smaller than fold variance.