Where does decoder collide?

In information theory a decoder is the receiver's stage that takes a coded, noise-corrupted signal and works back to the message the sender intended. Shannon's 1948 A mathematical theory of communication defined the channel model in which an encoder adds structured redundancy and a decoder uses that redundancy to correct errors and recover the original symbols. Viterbi's 1967 Error bounds for convolutional codes and an asymptotically optimum decoding algorithm gave the standard efficient algorithm for doing this on convolutional codes. The referent here is a rule or device that inverts a known encoding to retrieve a transmitted message, not a brain reading out movement from neurons or a network fabricating output from a compressed code.[1][2]

Unit

One decoding operation: a received code block mapped to its most likely source message

Recovery means

Recovery is the whole point: it restores the sent symbols despite channel noise, up to the code's error-correcting limit

Measured by

Judged by bit or block error rate, and by how close it comes to the channel capacity Shannon set as the limit

In neuroscience decoding means reading a stimulus, percept, or intended movement back out of the activity of many neurons at once. Georgopoulos, Schwartz, and Kettner's 1986 Neuronal population coding of movement direction showed that the direction a monkey reached could be recovered as a population vector summed across motor-cortex cells, none of which specified the direction alone. Bialek, Rieke, de Ruyter van Steveninck, and Warland's 1991 Reading a neural code then reconstructed a time-varying stimulus from a single sensory neuron's spike train. The referent is an estimate of what the world or the body is doing, inferred from measured firing, not a message pulled off a communication channel or the output side of an artificial network.[3][4]

Unit

One reconstruction: a stimulus value or movement estimate inferred from recorded spike activity

Recovery means

Recovers an external variable such as a reach direction or a stimulus from neural firing, within the accuracy the recorded population allows

Measured by

Measured by how well the estimate tracks the true stimulus or movement, for example decoding accuracy or reconstruction error

In machine learning a decoder is the half of an encoder-decoder network that turns a compact internal representation back into full output. Hinton and Salakhutdinov's 2006 Reducing the dimensionality of data with neural networks named the two halves of an autoencoder: an encoder that compresses data to a low-dimensional code, and a decoder that rebuilds the input from it. Cho and colleagues' 2014 Learning phrase representations using RNN encoder-decoder for statistical machine translation extended the idea so a decoder generates a whole target sequence from an encoded one. The referent is a trained mapping that synthesizes output from a learned latent code, not a receiver undoing channel noise or a readout of biological neurons.[5][6]

Unit

One forward pass of the decoder: a latent representation mapped to a generated output

Recovery means

Not applicable

Measured by

Assessed by reconstruction loss for autoencoders, or by output quality metrics such as translation BLEU for sequence decoders