Where does distillation collide?

Distillation separates the components of a liquid mixture by heating it so the more volatile parts boil off first and are then cooled back to liquid, using the differences in boiling point. The practice is ancient, and Hieronymus Brunschwig's 1500 Liber de arte distillandi, the first printed book devoted to the art, gathered the apparatus and procedures then in use; Forbes's 1970 A Short History of the Art of Distillation traces its development from early alchemy to industrial columns. Here the referent is a physical separation of matter by vaporisation and condensation, which has nothing in common with training a small model to copy a large one or with concentrating quantum entanglement, beyond the shared picture of drawing something pure out of a mixture.[1][2]

Unit

One separation of a liquid mixture into fractions ordered by boiling point

Recovery means

Recovery is the proportion of a target component captured in the collected distillate

Measured by

Followed by boiling point, the composition of vapour and liquid, and a column's reflux ratio or number of theoretical plates

Entanglement distillation takes many pairs of particles that share weak or noisy quantum entanglement and, using only local operations and classical communication, turns them into a smaller number of pairs whose entanglement is almost perfect. Bennett, Brassard, Popescu, Schumacher, Smolin, and Wootters introduced it in their 1996 Purification of Noisy Entanglement and Faithful Teleportation via Noisy Channels, and Bennett, DiVincenzo, Smolin, and Wootters set out the general theory the same year in Mixed-state entanglement and quantum error correction. The referent is a protocol acting on physical quantum states to raise the quality of entanglement, unrelated to boiling a liquid apart or shrinking a neural network, though all three meanings borrow the idea of concentrating a pure product.[3][4]

Unit

One round of local operations turning many noisy pairs into fewer near-maximally entangled pairs

Recovery means

Not applicable

Measured by

Quantified by the fidelity of the output pairs to a maximally entangled state and the yield of pairs per input

Knowledge distillation trains a small, fast student model to reproduce the behaviour of a large, cumbersome teacher model, learning from the teacher's softened output probabilities rather than the raw labels alone. Bucilua, Caruana, and Niculescu-Mizil originated the idea as model compression in their 2006 Model compression, transferring the predictions of a large ensemble into a single small network; Hinton, Vinyals, and Dean generalised it and gave it the name distillation in their 2015 Distilling the Knowledge in a Neural Network. The referent is a transfer of learned capability from one statistical model to a smaller one, which shares nothing with separating a chemical mixture or purifying quantum entanglement except the metaphor of extracting a concentrated essence.[5][6]

Unit

One student model trained to match a teacher model's softened output probabilities

Recovery means

Not applicable

Measured by

Assessed by the student's accuracy relative to the teacher and its reduction in size or compute