Leakage, Drift and Model Operations

1Leakage: training on the future

Leakage is any way the training pipeline sees information that will not exist at prediction time — the features the fitted model trained on are not drawn from the information set the deployed model will actually have when it scores a new case.

Three mechanisms cover almost every instance. Target leakage: a column recorded at or after the outcome, like 'date account closed' predicting churn. Preprocessing leakage: statistics fitted on all rows before the split, so the scaler or vocabulary has already seen the test set. Temporal leakage: shuffling time-ordered rows, so the model trains on Tuesday to predict Monday.

The symptom is a validation score too good for the difficulty of the problem, followed by production numbers that look like a different model — because they are: the deployed model is the same weights minus the information that never existed at scoring time.

Figure. Time runs left to right and the dashed cut is the moment the deployed model must score. Every feature must sit wholly left of the cut; the outcome and anything recorded with it sit right. A right-side column used as a feature is target leakage, whatever its correlation looks like.

2Pipelines that cannot leak

Leakage is prevented by construction, not by vigilance. The construction: split first, then treat everything fitted — scalers, vocabularies, encoders, imputers — as part of the model, fitted on training rows only and merely applied to validation and test.

This is what a pipeline object is for: bundle preprocessing and model, so cross-validation refits the whole chain inside each fold and the test set stays unseen by every fitted component. For time-ordered data add the temporal rule — validation rows postdate training rows, always.

The column-level check completes it: for every feature, ask at what moment its value becomes knowable. Any answer at or after the outcome disqualifies the column, however predictive it looks — especially however predictive it looks.

The hygiene, in order

1. Split firstPartition rows — by time, when time exists — before any statistic is computed.
2. Fit insideScalers and encoders are model parts: fitted on train folds only, applied elsewhere.
3. Interrogate columnsPer feature: when is this knowable? At or after the outcome means it goes.

Coding lab. Watch a scaler leak runs in the app, with checks on your output.

3Distribution shift: the world moves

Every offline score is measured on a frozen sample, and the population a deployed model scores keeps moving. Covariate shift: the inputs' distribution changes — a new customer mix — while the input-to-outcome relation holds. Label shift: the outcome rate changes — fraud doubles — while what fraud looks like does not. Concept drift: the relation itself changes — fraudsters adapt — and it is the hardest, because yesterday's labels are now wrong about today.

The practical consequence: an unchanged offline metric is not evidence of an unchanged model-world fit. The number is frozen at its sample date, and only quantities recomputed on live traffic can notice a move.

4The model shapes its own data

Once a model acts, it edits the dataset its successor will train on. A fraud model that blocks a transaction erases the label you needed — would it have been fraud? A credit model that declines an applicant never observes whether they would have repaid. This is the selective-labels problem: the outcomes you can see are conditioned on yesterday's model having said yes.

Some loops amplify instead of censor: send patrols where predicted crime is high and recorded incidents rise there, which raises the next model's prediction for the same place. The model's output has become its own input; recommenders do the same thing with taste.

The standard countermeasure is to keep a small randomised slice of decisions outside the model's control — approve a random sliver, review a random sample — so an unconditioned stream of labels keeps arriving. It costs a little in the short run, and it is the only window left open.

Figure. The loop runs clockwise: the model scores, decisions act, outcomes get labelled, labels refit the next model. The dashed chord is the defect — declined cases skip the labelling station entirely, so the training-data node holds only what past models said yes to.

Keep a window open

1. Name the censoringFor each decision, write down which outcomes it prevents you from ever observing.
2. Hold out at randomRoute a small random share of cases past the model, and label all of them.
3. Compare streamsModel-approved versus randomly-approved outcomes measure the loop's size.

5Deployment is not the end

Monitoring a model means recomputing, on live traffic, everything you trusted offline: input distributions against their training profile, the score distribution, the live decision rate, outcome rates as labels mature, and every slice metric the fairness audit ran. A metric computed only at launch is a metric you no longer have.

Alerts attach to divergence, not to thresholds picked by mood. The live flag rate halving while offline accuracy sits still is the signature of drift, and a fraud model whose flag volume quietly collapses looks cheaper and better right up until the chargebacks arrive. Green for the wrong reason is the failure monitoring exists to name.

Figure. Recompute on live traffic what you trusted offline: inputs against the training profile, score distribution and decision rate, then headline and slice outcomes as labels mature. Green for the wrong reason is the failure monitoring exists to name.

What to watch

1. InputsCompare live feature distributions to the training profile on a schedule.
2. BehaviourTrack score distribution and decision rate; step changes precede metric changes.
3. Outcomes, slicedAs labels mature, recompute headline and slice metrics on live cohorts.

6Rollback, owners and the paper trail

A model in production is an operational component and gets the discipline of one: a named owner who is paged when it misbehaves, a versioned artifact — weights plus feature schema plus a reference to the training-data snapshot — that can be redeployed in minutes, and a written rule for when to pull it, decided before launch, while nobody is defending a launch.

Rollback is cheap exactly when the previous version is kept warm and the feature schema is versioned with the model; it is impossible when 'the model' is a pile of weights whose input contract nobody recorded. The decision log — what data, which metrics, who approved — is what turns an incident into a fix instead of an argument.