A Model That Is 99% Accurate Can Be Completely Useless

Here is a model that detects fraudulent transactions with 99.8% accuracy. It is better than almost anything you will build in your first year. It took one line to write, it needs no training data, and it is worthless.

The model is: predict that every transaction is legitimate.

If two transactions in a thousand are fraudulent, that model is right 99.8% of the time and catches no fraud at all. The number is real, the model is useless, and the gap between those two facts is most of what makes machine learning harder than it looks.

The tools are not the difficulty. Fitting a model is a few lines, and the libraries are excellent. The difficulty is knowing whether the result means anything — and most of the ways it can fail to mean anything do not announce themselves.

What machine learning actually is

Conventional programming is rules in, answers out. You write the logic, the computer applies it.

Machine learning inverts that: examples in, rules out. You supply inputs paired with correct outputs, and an algorithm searches for a function that maps one to the other well enough to work on examples it has never seen.

That last clause is the entire discipline. A model that reproduces its training data perfectly has achieved nothing — a lookup table does that. The goal is generalisation: performance on data the model has not seen. Everything else in this article is a way of protecting that goal from the many things that quietly undermine it.

It also tells you when not to use machine learning at all. If the rule can be written down, write it down. A model that learns "reject orders over the credit limit" from ten thousand examples is strictly worse than an if-statement: harder to audit, harder to change, and occasionally wrong.

Accuracy is the wrong metric more often than not

Accuracy is the fraction of predictions that were correct, and it is almost always the first number people report. It fails whenever the classes are imbalanced, which is most interesting problems: fraud, disease, equipment failure, churn.

The replacements worth knowing are few and not difficult.

Precision asks: of the cases we flagged, how many were real? Low precision means false alarms, which cost the investigation effort and the user's goodwill.

Recall asks: of the real cases, how many did we catch? Low recall means misses, which cost whatever the thing you failed to catch costs.

These trade against each other, and the trade is a business decision rather than a technical one. A cancer screening test should favour recall, because a missed case is catastrophic and a false alarm leads to a second test. A system that automatically blocks accounts should favour precision, because wrongly locking out a paying customer is expensive and visible.

The crucial consequence is that you cannot choose a metric without knowing what the errors cost. An engineer who optimises a model without asking what happens to a false positive is optimising something arbitrary.

The baseline you must beat

Before any model, establish what trivial performance looks like. Always predict the majority class. Always predict the average. Use last week's value. Use the simple rule the business already applies.

This takes twenty minutes and it is the most informative twenty minutes in the project, because it converts "the model gets 85%" — a number with no meaning — into "the model gets 85% where guessing gets 84%", which is a verdict.

A surprising number of production models do not beat their baselines. They were never compared, so nobody found out.

Data leakage, the failure that looks like success

Leakage is when information that would not be available at prediction time leaks into training. It is the most common serious mistake in applied machine learning, and its signature is a model that performs implausibly well.

The forms it takes are varied and all look innocent. A feature computed after the outcome — number of support tickets about a cancellation, used to predict cancellation. Scaling the data using statistics computed over the whole dataset before splitting, so the training set knows the test set's mean. Splitting a time series randomly, so the model trains on the future and predicts the past. Duplicate rows that land in both splits. An identifier that happens to correlate with the target because records were created in order.

The discipline that prevents most of it: split first, then do everything else inside the training set only. Any transformation that learns something from the data — scaling, encoding, imputing, selecting features — must learn it from training data and be applied unchanged to the test set.

And the heuristic that catches the rest: if a result seems too good, it is leakage until proven otherwise. That instinct is worth more than any validation technique.

Overfitting, underfitting, and what to do about each

An overfitted model has learned the training data including its noise. It performs excellently in training and poorly on anything new. An underfitted model has not learned enough to perform well anywhere.

You can tell them apart by comparing training and validation performance. A large gap means overfitting; poor performance on both means underfitting. That one comparison directs almost all the decisions that follow.

For overfitting: more data, fewer features, a simpler model, or regularisation, which penalises complexity during training. For underfitting: a more capable model, better features, or longer training.

Underfitting is easier to spot and easier to fix. Overfitting is the one that reaches production, because it looked excellent during development.

Validation, and why a single split is not enough

A single train-test split gives one estimate, and that estimate has variance — split differently and the number moves. Cross-validation averages over several splits and gives a far more stable picture, along with a sense of how much the result depends on which rows ended up where.

Two cases need special handling.

Time series must be split by time. A random split lets the model learn from the future, which is both leakage and a guarantee that the offline result will not survive deployment.

Grouped data must be split by group. Several readings from the same patient, several sessions from the same user, several photographs of the same object — if rows from one group appear in both splits, the model can recognise the group rather than learn the pattern.

And the test set is spent the moment you use it to make a decision. If you check test performance, change something, and check again, you have begun fitting to it. That is what the validation set is for; the test set should be touched once, at the end.

Features usually matter more than the algorithm

Beginners spend their time choosing between algorithms. Practitioners spend theirs on features, because that is where the returns are.

A feature encodes domain knowledge in a form the model can use. A raw timestamp is nearly useless; hour of day, day of week and whether it is a holiday are informative. A raw address is useless; distance to the nearest branch is not. The difference between an adequate model and a good one is usually three well-chosen features rather than a different algorithm.

This is also why domain knowledge is not optional. Someone who understands how the business works will propose features a pure modeller never would, and the model will be better for reasons no hyperparameter search could find.

Start simple, and often stop there

Logistic regression and gradient-boosted trees between them handle a very large share of real tabular problems. Trees in particular are a strong default: they handle mixed types, need little preprocessing, cope with non-linear relationships, and are hard to beat on structured data.

Deep learning dominates images, audio and text. For a table of customer records it is usually worse, needs far more data, and takes longer to train.

The reason to start simple is not modesty. A simple model trains in seconds, so you can iterate on the things that matter — features, validation, the metric. A complex model takes hours, which means fewer experiments and slower learning. Establish that the problem is solvable before adding capacity.

The things that break after deployment

A model is not finished when it performs well offline. Several failures only appear in production.

Distribution shift: the world changes and the data stops resembling what the model trained on. Prices change, behaviour changes, a competitor launches. Performance degrades quietly, with no error raised.

Training-serving skew: a feature computed one way in the training pipeline and slightly differently in production. This is extremely common and extremely hard to detect, because both pipelines work.

Feedback loops: the model changes the behaviour it is predicting. A recommender that promotes popular items makes them more popular, and the training data for the next version is the output of the previous one.

The response is monitoring the inputs as well as the outputs. Watching accuracy is not enough, because labels often arrive late or never. Watching whether the incoming feature distributions resemble the training distributions catches the problem earlier.

Fairness is a technical problem with a non-technical core

A model trained on historical decisions learns the historical pattern, including the parts nobody would endorse. If past hiring favoured a group, a model trained on it will reproduce that and call it a prediction.

Removing the sensitive attribute does not fix it, because other features correlate with it — postcode with ethnicity, certain purchases with gender, employment gaps with caring responsibilities. The model reconstructs what you removed.

There are measures of fairness, and importantly they are mutually incompatible in general: you cannot satisfy all of them at once, so choosing one is a value judgement rather than an optimisation. That is uncomfortable, and it is better to make the choice explicitly than to let the default optimisation make it silently.

Preprocessing belongs inside the pipeline

Scaling, encoding categories, imputing missing values and selecting features

are all steps that learn something from the data, and every one of them is a

leakage risk if fitted before the split.

The structural fix is to express the whole sequence as a pipeline object that

is fitted as a unit. Cross-validation then refits the entire chain on each

fold, so the scaler sees only that fold's training data, and the result is an

honest estimate rather than an optimistic one.

It also removes a second, quieter failure: the mismatch between what happened

during training and what happens at prediction time. If preprocessing lives in

a notebook cell rather than in the model artefact, somebody has to reproduce it

exactly in the serving code, and small differences in how a category is encoded

or a missing value filled will degrade the model in ways that are very hard to

trace. A pipeline that carries its own preprocessing cannot drift apart from

itself.

Hyperparameters, and spending the search budget sensibly

Most models have settings that are not learned from data: tree depth,

regularisation strength, learning rate, number of estimators.

Exhaustive grid search is the obvious approach and usually the wrong one,

because the parameters are not equally important. A grid spends the same effort

on a parameter that barely matters as on the one that dominates. Random search

over sensible ranges generally finds better results in the same time, for the

simple reason that it tries more distinct values of the parameter that matters.

More important than the search method is what you search against. Tuning

against the test set, even indirectly through repeated checking, inflates the

final estimate. The honest structure is a training set for fitting, a

validation set for every decision including hyperparameters, and a test set

touched exactly once.

And it is worth keeping the returns in perspective. Tuning typically buys a

small improvement. Better features, more data, or correcting a leak typically

buy a large one, and they should be exhausted first.

Document the model, not just the code

A model that reaches production acquires obligations the code does not. Six

months later somebody will ask what it was trained on, which population it was

validated against, what its known weaknesses are, and whether it is still

appropriate for a use case nobody anticipated.

Writing that down at the time costs an hour and is almost never done. The

useful contents are short: what the model predicts and what it does not, the

data it was trained on and the period it covers, the metric it was optimised

for and why that metric was chosen, the groups it was evaluated across, and the

conditions under which it should not be used.

The last item is the one with the most value and the least precedent. A model

built to rank support tickets by urgency will eventually be proposed for

deciding staffing levels, and the person proposing it will have no way to know

that it was never validated for that unless somebody said so in writing.

What to practise

1. Compute the baseline first on any problem, before building anything.
2. Build a model with deliberate leakage — include a feature from after the outcome — and see how good the numbers look. The lesson is in the plausibility.
3. Take an imbalanced dataset and report accuracy, precision and recall. Decide which error is worse before looking at any of them.
4. Split a time series randomly, then chronologically, and compare. The gap is the size of the mistake.
5. Spend a day on features instead of algorithms and compare the improvement to what a different model gave you.

Machine learning rewards suspicion of your own results. The discipline is not fitting models — that part is easy and getting easier. It is constructing an evaluation you can believe, and most of the skill is knowing the specific ways that evaluation can lie.