A model is fitted to a world that then moves. Nothing in the model notices, and nothing in its output says so - a drifted model returns confident predictions at the same rate as a healthy one. Monitoring is the only thing that catches it.
The two kinds of drift
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
rng = np.random.default_rng(3)
n = 4000
X_train = rng.normal(0, 1, (n, 3))
y_train = ((1.2 * X_train[:, 0] - 0.8 * X_train[:, 1]
+ rng.normal(0, 0.9, n)) > 0).astype(int)
model = Pipeline([('s', StandardScaler()),
('m', LogisticRegression(max_iter=1000))]).fit(X_train, y_train)
def auc(X, y):
return roc_auc_score(y, model.predict_proba(X)[:, 1])
# No drift.
X_same = rng.normal(0, 1, (n, 3))
y_same = ((1.2 * X_same[:, 0] - 0.8 * X_same[:, 1] + rng.normal(0, 0.9, n)) > 0).astype(int)
# COVARIATE drift: the inputs moved, the rule did not.
X_shifted = rng.normal(1.8, 1.4, (n, 3))
y_shifted = ((1.2 * X_shifted[:, 0] - 0.8 * X_shifted[:, 1]
+ rng.normal(0, 0.9, n)) > 0).astype(int)
# CONCEPT drift: the inputs look the same, the rule changed.
X_concept = rng.normal(0, 1, (n, 3))
y_concept = ((-0.4 * X_concept[:, 0] + 1.3 * X_concept[:, 2]
+ rng.normal(0, 0.9, n)) > 0).astype(int)
print(f'{"no drift":<24} AUC {auc(X_same, y_same):.3f}')
print(f'{"covariate drift":<24} AUC {auc(X_shifted, y_shifted):.3f}')
print(f'{"concept drift":<24} AUC {auc(X_concept, y_concept):.3f}')
print()
print('Covariate drift is visible in the INPUTS without any labels.')
print('Concept drift is not - the inputs look normal, and only the')
print('outcomes reveal it.')
Open this example in the editor and change it. Nothing you type is saved.
- Covariate drift: the distribution of the inputs changes. A new marketing channel brings different customers; a new device becomes popular. Detectable without labels, by watching the features.
- Concept drift: the relationship between inputs and label changes. A competitor launches, a regulation changes, a fraud ring adapts. Not detectable from the inputs at all - only outcomes reveal it.
That asymmetry decides the monitoring strategy: input monitoring is cheap and immediate but blind to the worse failure; outcome monitoring catches everything and arrives late.
Detecting covariate drift
import numpy as np
import pandas as pd
rng = np.random.default_rng(7)
def psi(expected, actual, bins=10):
"""Population Stability Index. Under 0.1 is stable, 0.1-0.25 needs
watching, above 0.25 is a real shift."""
edges = np.percentile(expected, np.linspace(0, 100, bins + 1))
edges[0], edges[-1] = -np.inf, np.inf
e = np.histogram(expected, bins=edges)[0] / len(expected)
a = np.histogram(actual, bins=edges)[0] / len(actual)
e, a = np.clip(e, 1e-6, None), np.clip(a, 1e-6, None)
return float(np.sum((a - e) * np.log(a / e)))
reference = rng.normal(0, 1, 5000)
cases = {
'identical': rng.normal(0, 1, 5000),
'mean +0.2': rng.normal(0.2, 1, 5000),
'mean +0.5': rng.normal(0.5, 1, 5000),
'mean +1.0': rng.normal(1.0, 1, 5000),
'wider spread': rng.normal(0, 1.8, 5000),
}
for name, sample in cases.items():
value = psi(reference, sample)
verdict = 'stable' if value < 0.1 else ('watch' if value < 0.25 else 'SHIFTED')
print(f'{name:<14} PSI {value:.3f} {verdict}')
Open this example in the editor and change it. Nothing you type is saved.
Three signals worth monitoring on the inputs, in order of usefulness:
import numpy as np
import pandas as pd
rng = np.random.default_rng(11)
reference = pd.DataFrame({
'amount': rng.gamma(3, 40, 4000),
'country': rng.choice(['GB', 'US', 'FR'], 4000, p=[.6, .3, .1]),
'age_days': rng.gamma(2, 120, 4000),
})
live = pd.DataFrame({
'amount': rng.gamma(3, 40, 4000),
'country': rng.choice(['GB', 'US', 'FR', 'DE'], 4000, p=[.4, .3, .1, .2]),
'age_days': rng.gamma(2, 120, 4000),
})
live.loc[rng.choice(4000, 320, replace=False), 'amount'] = np.nan
print('1. missing rates')
for column in reference.columns:
before = reference[column].isna().mean()
after = live[column].isna().mean()
flag = ' <- CHANGED' if abs(after - before) > 0.02 else ''
print(f' {column:<10} {before:.1%} -> {after:.1%}{flag}')
print()
print('2. new or vanished categories')
before_set = set(reference['country'])
after_set = set(live['country'])
print(f' new: {sorted(after_set - before_set)}')
print(f' vanished: {sorted(before_set - after_set)}')
print()
print('3. the predicted-score distribution, which needs no labels')
print(' (a shift here is the earliest warning you can get)')
Open this example in the editor and change it. Nothing you type is saved.
The third one is the most valuable: the distribution of the model's own output. It needs no labels, it moves when either kind of drift begins, and a single number - the mean predicted probability - is often enough.
Detecting concept drift
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
rng = np.random.default_rng(13)
X_train = rng.normal(0, 1, (4000, 3))
y_train = ((1.2 * X_train[:, 0] - 0.8 * X_train[:, 1] + rng.normal(0, 0.9, 4000)) > 0).astype(int)
model = Pipeline([('s', StandardScaler()),
('m', LogisticRegression(max_iter=1000))]).fit(X_train, y_train)
rows = []
for week in range(1, 13):
X = rng.normal(0, 1, (900, 3))
# From week 6 the rule starts changing.
strength = 1.2 if week < 6 else 1.2 - 0.35 * (week - 5)
y = ((strength * X[:, 0] - 0.8 * X[:, 1] + rng.normal(0, 0.9, 900)) > 0).astype(int)
probabilities = model.predict_proba(X)[:, 1]
rows.append({
'week': week,
'mean_input': round(X.mean(), 3),
'mean_score': round(probabilities.mean(), 3),
'auc': round(roc_auc_score(y, probabilities), 3),
})
table = pd.DataFrame(rows)
print(table.to_string(index=False))
print()
print('mean_input never moves - the covariate monitor sees nothing.')
print(f'AUC falls from {table["auc"].iloc[0]:.3f} to {table["auc"].iloc[-1]:.3f}.')
print('Only the labelled outcomes reveal it, and they arrive late.')
Open this example in the editor and change it. Nothing you type is saved.
Which means: you must measure the live outcome, and the delay before it arrives is the length of your blind spot. A churn model whose outcome is known 90 days later has a 90-day blind spot, and the only mitigation is to monitor a faster proxy alongside it.
What to monitor, in order of usefulness
import pandas as pd
print(pd.DataFrame([
{'signal': 'live outcome metric (AUC, precision at the threshold)',
'needs labels': 'yes', 'latency': 'as long as labels take',
'catches': 'everything'},
{'signal': 'predicted score distribution',
'needs labels': 'no', 'latency': 'immediate',
'catches': 'both kinds, indirectly'},
{'signal': 'input distributions (PSI per feature)',
'needs labels': 'no', 'latency': 'immediate',
'catches': 'covariate drift only'},
{'signal': 'missing rates and new categories',
'needs labels': 'no', 'latency': 'immediate',
'catches': 'upstream breakage'},
{'signal': 'volume and latency of the scoring service',
'needs labels': 'no', 'latency': 'immediate',
'catches': 'the pipeline being broken'},
]).to_string(index=False))
Open this example in the editor and change it. Nothing you type is saved.
The last row matters more than it looks. Most "the model stopped working" incidents are not drift at all - they are a renamed upstream column, a feature pipeline that failed, or an encoder meeting a category it has never seen.
Retraining is not always the answer
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
rng = np.random.default_rng(17)
def make(n, strength, seed):
generator = np.random.default_rng(seed)
X = generator.normal(0, 1, (n, 3))
y = ((strength * X[:, 0] - 0.8 * X[:, 1] + generator.normal(0, 0.9, n)) > 0).astype(int)
return X, y
def fit(X, y):
return Pipeline([('s', StandardScaler()),
('m', LogisticRegression(max_iter=1000))]).fit(X, y)
X_old, y_old = make(4000, 1.2, 1)
old_model = fit(X_old, y_old)
cases = {
'nothing changed': (1.2, 'noise - do not retrain'),
'small real change': (0.8, 'maybe - check it is worth the risk'),
'large real change': (-0.5, 'retrain, and investigate why'),
}
for label, (strength, advice) in cases.items():
X_new, y_new = make(4000, strength, 2)
old_auc = roc_auc_score(y_new, old_model.predict_proba(X_new)[:, 1])
new_model = fit(X_new, y_new)
new_auc = roc_auc_score(y_new, new_model.predict_proba(X_new)[:, 1])
print(f'{label:<20} old {old_auc:.3f} retrained {new_auc:.3f} '
f'gain {new_auc - old_auc:+.3f} {advice}')
print()
print('Retraining on the same amount of data has its own variance.')
print('A gain inside the noise is not a reason to deploy.')
Open this example in the editor and change it. Nothing you type is saved.
Four cases where retraining is the wrong response:
It was noise. A weekly metric moves by sampling variation; see the uncertainty lesson.
The pipeline broke. Retraining on broken data bakes the breakage in.
The world changed in a way the features cannot capture. A new competitor is not in your feature set, and retraining learns a worse version of the old relationship.
The labels are now wrong. If the labelling process changed, retraining teaches the model the new mistake.
The right first question is always: has the data changed, or has the world changed, or has the pipeline changed? They need three different responses.
Retrain on a schedule, deploy on evidence
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
rng = np.random.default_rng(19)
def window(start_week, weeks, strength_at):
X, y = [], []
for week in range(start_week, start_week + weeks):
Xw = rng.normal(0, 1, (400, 3))
s = strength_at(week)
yw = ((s * Xw[:, 0] - 0.8 * Xw[:, 1] + rng.normal(0, 0.9, 400)) > 0).astype(int)
X.append(Xw)
y.append(yw)
return np.vstack(X), np.concatenate(y)
strength_at = lambda week: 1.2 if week < 20 else max(1.2 - 0.09 * (week - 19), 0.1)
X_train, y_train = window(0, 12, strength_at)
production = Pipeline([('s', StandardScaler()),
('m', LogisticRegression(max_iter=1000))]).fit(X_train, y_train)
rows = []
for week in range(12, 34, 2):
X_eval, y_eval = window(week, 2, strength_at)
live_auc = roc_auc_score(y_eval, production.predict_proba(X_eval)[:, 1])
# A candidate retrained on the most recent 12 weeks.
X_recent, y_recent = window(max(week - 12, 0), 12, strength_at)
candidate = Pipeline([('s', StandardScaler()),
('m', LogisticRegression(max_iter=1000))]).fit(X_recent, y_recent)
candidate_auc = roc_auc_score(y_eval, candidate.predict_proba(X_eval)[:, 1])
deploy = candidate_auc - live_auc > 0.01
rows.append({'week': week, 'live_auc': round(live_auc, 3),
'candidate_auc': round(candidate_auc, 3),
'gain': round(candidate_auc - live_auc, 3),
'deploy': deploy})
if deploy:
production = candidate
print(pd.DataFrame(rows).to_string(index=False))
print()
print('A candidate is retrained every fortnight regardless, and')
print('deployed only when it beats the live model on the SAME recent')
print('data by more than the noise. That separates "we always have a')
print('fresh model ready" from "we change production".')
Open this example in the editor and change it. Nothing you type is saved.
The discipline in three lines:
- Retrain on a schedule, so a candidate always exists and the pipeline is always exercised.
- Deploy on evidence: the candidate must beat the live model on the same recent data, by more than the fold-to-fold noise.
- Keep the previous model, and be able to roll back in minutes.
The feedback loop
import numpy as np
import pandas as pd
rng = np.random.default_rng(23)
# A fraud model that BLOCKS what it flags. Blocked transactions never
# produce an outcome, so the training data loses exactly the cases the
# model is most confident about.
n = 20_000
latent = rng.normal(0, 1, n)
fraud = (latent > 1.8).astype(int)
score = latent + rng.normal(0, 0.4, n)
rows = []
for threshold in [np.inf, 2.5, 1.5, 0.8]:
blocked = score > threshold
observed = ~blocked
label = 'no blocking' if threshold == np.inf else f'block above {threshold}'
rows.append({
'policy': label,
'blocked': f'{blocked.mean():.1%}',
'fraud rate in observed data': f'{fraud[observed].mean():.2%}',
'true fraud rate': f'{fraud.mean():.2%}',
})
print(pd.DataFrame(rows).to_string(index=False))
print()
print('The more the model blocks, the cleaner the data it is retrained')
print('on looks - and the less it learns about the fraud it is already')
print('catching. Left alone, it forgets.')
Open this example in the editor and change it. Nothing you type is saved.
Three mitigations, and a system that acts on its own predictions needs at least one:
- Hold out a random control: let a small random percentage through unblocked, so unbiased labels keep arriving. This costs money and is usually worth it.
- Record the score at decision time with every outcome, so the bias is measurable rather than invisible.
- Watch the distribution of what you block, not only what you let through.
The same loop appears in recommendations (you only see clicks on what you showed), in hiring (you only see performance for who you hired), and in credit (you only see repayment for who you approved). It has a name - selection bias from the policy - and it is why a model that acts on the world needs a different monitoring design from one that only observes it.
When it goes wrong
| Symptom | Cause |
|---|---|
| Accuracy fell and the inputs look normal | Concept drift; only outcomes show it |
| The model stopped working overnight | Usually a broken pipeline, not drift |
| Retraining did not help | The world changed beyond the features |
| A weekly dip triggered a retrain | Noise; compare with the interval |
| The retrained model was worse | Retrained on data the policy had biased |
| An unseen category crashed scoring | Monitor new categories; use handle_unknown |
| Drift was found three months late | No outcome monitoring; add a proxy |
| Blocked cases vanished from training | Feedback loop; hold out a random control |
A check you can run
For the model you have in production, answer one question: how long after a prediction does the true outcome arrive, and is anyone comparing the two?
That delay is the length of your blind spot. If nobody is comparing, the blind spot is infinite - the model could have been predicting at chance for a month, returning confident probabilities the whole time, and the first signal would be a business metric nobody connected to it.