no free lunch theorem: a standalone construct

David Wolpert and William Macready introduced the no free lunch theorems for optimization in their 1997 paper of that name, proving that when performance is averaged over every possible objective function (the quantity an algorithm tries to maximise or minimise), no search or optimization method does better than any other, including blind random search. A year earlier Wolpert (1996) had established the parallel result for supervised machine learning, showing that no learning algorithm is inherently superior once its accuracy is averaged across all possible target functions. The underlying idea is that an algorithm can only gain an edge on some problems by giving it back on others, so any advantage must come from matching the algorithm to the structure of the specific problems it will face rather than from the algorithm on its own. The theorem is a formal, assumption-free statement about the limits of general-purpose optimization and learning: it holds by counting uniformly over the full space of possible problems, which is why it is invoked to argue that benchmark results cannot be extrapolated to arbitrary tasks and that some form of prior knowledge, also called inductive bias, is always required.[1][2]

Unit

A formal impossibility result (theorem) about the average performance of optimization and learning algorithms taken across the space of all possible problems

Recovery means

Not applicable

Measured by

Established by mathematical proof; its scope is tested by whether a claimed universal advantage survives averaging performance uniformly over all possible objective or target functions