overfitting: a standalone construct

In statistics and machine learning, overfitting is what happens when a model is flexible enough to fit the random noise and idiosyncrasies of its training data rather than the underlying pattern, so it scores well in-sample but generalises poorly to new data. It is the central reason models are evaluated on held-out data and penalised for excess complexity. Hawkins's The problem of overfitting lays out the problem and the standard safeguards. The referent is the failure mode of a model that is too complex for the evidence it was fit on.[1]

Unit

The gap between in-sample fit and out-of-sample performance

Recovery means

The criterion that exposes it is error on held-out data, not training error; it is reduced by simpler models, regularisation, more data, or cross-validation

Measured by

Diagnosed by a large gap between training and validation or test error, and by learning curves across model complexity