sparsity: one construct, many systems

In computational neuroscience, sparse coding is the idea that the brain represents each input using only a small number of simultaneously active neurons drawn from a large population, so that most units stay silent for any given stimulus. Olshausen and Field's 1996 Emergence of simple-cell receptive field properties by learning a sparse code for natural images showed that forcing a learning system to be sparse makes it discover edge-like receptive fields resembling those in primary visual cortex. The referent is an efficient neural representation in which few components carry the signal, the same few-matter principle that reappears in signal recovery and in economic attention.[1]

Unit

The fraction of units active per stimulus, kept low

Recovery means

Not applicable

Measured by

Quantified by activity sparseness (the share of near-zero responses) and by reconstruction quality under a sparsity penalty

In signal processing, sparsity is the premise behind compressed sensing: if a signal has only a few nonzero coefficients in some basis, it can be reconstructed exactly from far fewer measurements than classical sampling would demand, by seeking the sparsest explanation of the data. Donoho's 2006 Compressed sensing gave the conditions under which this exact recovery holds. The referent is the same structural fact (that the object of interest lives in a low-dimensional corner of a huge space), now exploited to sense and compress rather than to code neural activity.[2]

Unit

The number of nonzero coefficients needed to represent the signal

Recovery means

Not applicable

Measured by

Characterised by the sparsity level versus the number of measurements required for exact or stable recovery

In behavioral economics, sparsity models bounded rationality by assuming the decision-maker actively attends to only a few of the many variables that could matter and sets the rest to a default, deliberately simplifying the problem to save cognitive effort. Gabaix's 2014 A sparsity-based model of bounded rationality builds a tractable theory of such selectively inattentive agents. The referent is the same few-components-carry-the-weight idea, here governing which features of a choice a person actually processes.[3]

Unit

The number of variables an agent actively attends to

Recovery means

Not applicable

Measured by

Captured by attention weights on each variable, most shrunk toward zero relative to the full rational model