Where does quantization collide?

The principle that certain physical quantities, such as energy, angular momentum, and electric charge, can take only particular discrete values rather than any value along a continuum, together with the formal procedure for building a quantum theory from a classical description. Planck's 1901 Ueber das Gesetz der Energieverteilung im Normalspectrum introduced the idea that a radiating oscillator exchanges energy only in whole multiples of a small element, which is the historical root of the term. The referent is a fact about the physical world and a recipe for theory, not the amplitude rounding of a recorded signal or the precision cut applied to a neural network's numbers.[1]

Unit

A discrete allowed value of a physical quantity, in whole multiples of a fundamental constant

Recovery means

Not applicable

Measured by

Inferred from discrete spectral lines, energy levels, and quantised observables in experiment

The step in digitising a signal that maps a continuous or high-precision amplitude onto a finite set of discrete levels, so each sample is replaced by the nearest available level and the small leftover difference becomes quantization error, or quantization noise. Bennett's 1948 Spectra of Quantized Signals analysed the noise introduced by this rounding in pulse-code modulation and gave its statistical description. The referent is a deliberate loss of amplitude resolution in an engineered signal, not a discreteness built into nature nor the precision reduction of a model's weights.[2]

Unit

One sample mapped to its nearest level, spaced by a quantization step

Recovery means

Not applicable

Measured by

Quantified by step size, bit depth, and the signal-to-quantization-noise ratio

The engineering technique of reducing the numerical precision used to store and compute a neural network's weights and activations, for example replacing 32-bit floating-point numbers with 8-bit integers, so the model is smaller and faster while its accuracy is largely preserved. Jacob and colleagues' 2018 Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference set out a widely used scheme for integer-only inference and for training that anticipates the rounding. The referent is a chosen trade of precision for efficiency in a trained model, not the amplitude rounding of a recorded signal nor the discrete values that nature itself allows.[3]

Unit

A weight or activation stored at reduced bit width, such as an 8-bit integer

Recovery means

Fine-tuning, or quantization-aware training, can recover accuracy lost to the precision cut

Measured by

Assessed by bit width, model size, inference latency, and accuracy against the full-precision baseline