Where does residual collide?

In regression and curve fitting, a residual is what is left over after a model has been fitted: the difference between an observed data point and the value the model predicts for it. Anscombe and Tukey's 1963 The Examination and Analysis of Residuals showed that plotting and studying these leftovers, rather than only the fitted line, is how one checks whether a model's assumptions hold and where it fails. Here the residual is a per-observation number computed from data and a fitted model, used to diagnose that model, and it is not a share of economic growth or a wiring shortcut inside a neural network.[1]

Unit

One number per observation, in the units of the outcome being modelled

Recovery means

Not applicable

Measured by

Computed as observed minus fitted for each case, then inspected through residual plots and summed as a residual sum of squares

In growth accounting, the residual is the part of an economy's output growth that remains after the contributions of measured capital and labour have been subtracted, and it is read as a rough index of technological progress or total factor productivity. Solow's 1957 Technical Change and the Aggregate Production Function introduced this leftover term and the method that isolates it, and it is often described as a measure of our ignorance because it captures everything the input measures miss. Here the residual is an aggregate rate of growth attributed to no measured input, not a per-observation fitting error or a connection between layers of a network.[2]

Unit

A rate of growth per period, the share of output growth left unattributed to inputs

Recovery means

Not applicable

Measured by

Estimated in growth accounting as the difference between output growth and the input-weighted growth of capital and labour

In deep neural networks, a residual connection is a shortcut that adds a block's input directly to its output, so the stacked layers only have to learn the change to make to the input rather than a whole new mapping from scratch. He, Zhang, Ren, and Sun's 2016 Deep Residual Learning for Image Recognition introduced these connections in ResNet, which let very deep networks train without their accuracy degrading. Here the residual is an architectural wiring choice, the function the block learns on top of an identity shortcut, and it has nothing to do with a fitting error left in data or an unexplained portion of economic growth.[3]

Unit

One skip connection per block, adding the block's input to its transformed output

Recovery means

Not applicable

Measured by

Realised in a network as an identity shortcut summed with a block's output, so the block learns the residual function

In cultural theory, residual refers to cultural elements formed in the past that remain active in the present beside the dominant and the emergent. This meaning comes from Williams (1977) in Marxism and literature.[4]

Unit

Recovery means

Measured by