Where does entropy collide?

A property of a body in equilibrium, measured in joules per kelvin, whose change between two states equals the integral of heat absorbed divided by absolute temperature along any reversible path between them. Clausius fixed it in 1865, built the word from the Greek for transformation so it would sound like energy, and glossed it as the body's Verwandlungsinhalt, its transformation content. The second law in his final form is that the entropy of the universe tends to a maximum. Nothing in the definition mentions atoms, probability or disorder: entropy here is a number computed from a calorimeter and a thermometer, and it exists whether or not anyone knows what the body is made of.[1]

Unit

A body or closed system in equilibrium, between two states

Recovery means

Not applicable, a reversible cycle returns the system to its starting entropy; an isolated system's entropy never falls

Measured by

Joules per kelvin; heat capacity integrated over temperature from a reference state; tabulated standard molar entropies

The logarithm of the number of microscopic arrangements of a system's molecules compatible with its macroscopic state, times a constant that converts the count into Clausius's units. Boltzmann established the relation in 1877 by counting the ways energy could be distributed among gas molecules and showing that the most probable distribution is the equilibrium one, so that the second law becomes a statement about probability rather than a separate axiom; the compact S = k log W is Planck's later notation and is carved on Boltzmann's grave. Gibbs generalised it in 1902 to an ensemble average, minus the sum of probability times log probability over microstates, which reduces to Boltzmann's count when the microstates are equiprobable. The distinctive move is that entropy now depends on how the macrostate is specified: coarsen the description and the count of compatible microstates changes.[2][3]

Unit

A macrostate, defined by a chosen set of macroscopic variables, and the microstates compatible with it

Recovery means

Not applicable, the criterion is whether the entropy of the macrostate is at its maximum given the constraints

Measured by

Boltzmann's constant times the logarithm of the microstate count; the Gibbs ensemble sum over a partition function

The average uncertainty of a random source, minus the sum of each symbol's probability times its log probability, measured in bits when the log is base two. Shannon fixed it in 1948 as the unique measure satisfying continuity, monotonicity in the number of equiprobable outcomes and additivity over successive choices, and proved that it is the minimum average number of bits per symbol at which the source can be encoded without loss. Tribus and McIrvine record Shannon saying that von Neumann told him to call the quantity entropy because it was already used in statistical mechanics under that name and because nobody knew what entropy really was, so he would always have the advantage in a debate. Jaynes closed the loop in 1957: statistical mechanics is Shannon's problem run backwards, and the Gibbs distribution is the one that maximises entropy subject to what is known. Entropy here belongs to a probability distribution, not to a body, and has no temperature in it.[4][5][6]

Unit

A probability distribution over messages or symbols

Recovery means

Not applicable, the criterion is the coding rate the source admits, and the entropy is its floor

Measured by

Bits or nats computed from symbol frequencies; estimated from empirical distributions of a source

Two working quantities built from Shannon's. Cross-entropy is the expected log loss when a model's distribution q is used to encode data drawn from p; minimising it over the training set is maximum-likelihood estimation, and it is the default classification objective. Kullback and Leibler's 1951 divergence is cross-entropy minus the entropy of p, the extra bits paid for using the wrong distribution, and it is what variational methods minimise. Entropy also appears on the other side of the equation as a bonus: Haarnoja and colleagues' soft actor-critic of 2018 adds the expected entropy of the policy to the reward, weighted by a temperature, so the agent is paid to stay as random as it can while still succeeding. This is the AI spoke: entropy is a scalar to push down when it measures fit and to push up when it measures exploration, and the same word names both.[7][8]

Unit

A model's predictive distribution against data, or a policy's action distribution

Recovery means

Not applicable, the loss falls toward the entropy of the data and stops; the exploration bonus is annealed away

Measured by

Cross-entropy in nats or bits per example; KL divergence between two distributions; mean policy entropy per step

Shannon's formula applied to a community: p_i is the proportion of individuals belonging to species i, and H′ rises with the number of species and with how evenly individuals are spread across them. MacArthur brought the expression into ecology in 1955 as a measure of how many pathways energy could take through a food web, and hence of community stability. Pielou fixed the diversity reading in 1966, distinguished the versions appropriate to a finite collection, a sample and an infinite population, and defined evenness as H′ divided by its maximum, the log of species number. The formula is Shannon's to the symbol, but the referent is not: an ecologist who reports H′ is describing how a community's individuals are distributed, not how many bits a message needs, and two communities with the same H′ may share no species at all.[9][10]

Unit

A community, collection or sample of individuals sorted into species

Recovery means

Return of diversity after disturbance, H′ and evenness climbing back toward pre-disturbance values

Measured by

H′ from species counts in a sample; Pielou's J′ for evenness; compared across sites, seasons or treatments

The claim that the economic process is a physical one: it takes in low-entropy matter and energy, delivers them as goods, and returns them to the environment as high-entropy waste, so that the second law, not preferences or technology, sets the ultimate constraint on growth. Georgescu-Roegen fixed the argument in 1971 and attacked the circular-flow model for ignoring the arrow of time. Ecological economics took the book as a founding text. The reading is contested on its physics: Kåberger and Månsson showed in 2001 that the Earth is an open system fed by solar low entropy, that the entropy law does not by itself bound recycling or economic activity, and that the useful content of the thesis is about resource stocks and time, not thermodynamics. Entropy here is borrowed from Clausius as an argument, not computed as a quantity.[11][12]

Unit

An economy or the biosphere, treated as a system exchanging matter and energy with its surroundings

Recovery means

Not applicable, the thesis is that low-entropy stocks are not restored on human timescales; the critique is that solar inflow restores them continuously

Measured by

Rarely; resource and energy throughput, exergy accounts and material flows stand in for it