Where does convolution collide?

Convolution is an operation that takes two functions and produces a third, written f * g, which expresses how the shape of one is blended with the other as one is slid across it and the overlapping area is summed. In the study of linear time-invariant systems it is the operation that turns any input into the output once the system's impulse response is known, so that impulse response fully describes the system. The convolution theorem ties it to the frequency domain: convolving two functions in time corresponds to simply multiplying their transforms in frequency. Oppenheim, Willsky, and Nawab's Signals and Systems develops the convolution sum and integral, the impulse response, and this link to the Fourier domain. The referent is an abstract operation on functions, not a filter learned from images, a rule for adding random variables, or a fold of brain tissue.[1]

Unit

An operation on two functions, written f * g, giving a third function

Recovery means

Not applicable

Measured by

Computed as a sum or integral of one function times a shifted, reflected copy of the other, or as multiplication of their transforms

If two random variables are independent, the distribution of their sum is the convolution of their two individual distributions: to find the chance that the sum takes a given value, you add up, over all the ways the two parts can combine to that total, the product of their separate probabilities. The same rule holds for probability densities in the continuous case, with the sum replaced by an integral. This is why repeatedly adding independent copies of a variable smooths the distribution out and, under broad conditions, drives it toward the bell curve. Grimmett and Stirzaker's Probability and Random Processes presents the convolution formula for the distribution of a sum of independent variables. The referent is a rule for combining distributions, not an operation studied for its own sake, a filter over pixels, or a ridge of brain tissue.[2]

Unit

A probability distribution for the sum of independent variables

Recovery means

Not applicable

Measured by

Obtained by convolving the component distributions, summing or integrating over the ways the parts add to each total

In a convolutional neural network, convolution names a small array of weights, a filter or kernel, that is slid across an image so the same weights are applied at every position; each such filter produces a feature map that responds to a local pattern like an edge or a texture. Because the weights are shared across all positions and learned from data, the network can pick out a feature wherever it appears while using far fewer parameters than a fully connected layer would. LeCun, Bottou, Bengio, and Haffner's Gradient-Based Learning Applied to Document Recognition set out this architecture and used it to read handwritten digits. Strictly, the operation used in most such networks is cross-correlation, since the filter is not flipped as a true mathematical convolution would be, but the name has stuck. The referent is a learnable local image filter, not the abstract operation on functions, the sum-of-variables rule, or the folds of the cortex.[3]

Unit

A learned filter's feature map, one response value per image position

Recovery means

Not applicable

Measured by

Read off as feature maps and the learned filter weights, and judged by the network's accuracy on held-out images