Where does gesture collide?

The spontaneous hand and arm movements that people produce while speaking, timed with and organised around the flow of talk so that they help carry meaning alongside the words. Efron's 1941 Gesture and Environment was the founding empirical study of these movements, showing that they follow cultural patterns and travel with speech rather than replacing it. McNeill's 1992 Hand and Mind refined the account, treating the movement and the spoken utterance as two parts of a single thought produced together. The referent here is a bodily movement that accompanies talk, recorded frame by frame from video, not an input read by a machine, a communicative move by an ape, or an expressive shaping of a musical line.[1][2]

Unit

One hand or arm movement made during speech, segmented into preparation, stroke, and retraction

Recovery means

Not applicable

Measured by

Coded frame by frame from video by movement type, form, and timing against the accompanying words

A movement of the hand or body, such as a touch, swipe, point, or drawn shape, that a computer detects and treats as a command. Bolt's 1980 Put-that-there introduced the idea of driving a graphics interface by pointing and speaking together, letting a person move objects on a screen by gesturing at them. Wobbrock, Morris, and Wilson's 2009 User-defined gestures for surface computing is a widely used later study that asked ordinary people which movements they would choose for common commands on a touch surface. The referent here is an input recognised by a system and mapped to an action, defined by the sensor and the software, not a movement that accompanies speech, a signal exchanged between apes, or a felt motion in music.[3][4]

Unit

One recognised input movement mapped to a single command

Recovery means

Not applicable

Measured by

Captured by touchscreens, cameras, or motion sensors and classified by recognition algorithms

A communicative bodily movement, such as a reach, a touch, or a raised arm, that an ape or monkey directs at another individual to prompt a response. Tomasello, George, Kruger, Farrar, and Evans's 1985 study of young chimpanzees opened the study of these movements as learned, flexibly used signals rather than fixed reflexes. Tomasello and Call's 2019 Thirty years of great ape gestures reviews the large body of work that followed and how these signals are catalogued and interpreted. The referent here is a deliberate signal made by one nonhuman primate to another, recorded through observation, not a movement that accompanies human speech, an input read by a machine, or a shaping of a musical line.[5][6]

Unit

One communicative movement directed by one animal at another

Recovery means

Not applicable

Measured by

Recorded by field and captive observation and catalogued by form, context, and the response it elicits

An expressive unit of motion in music: a shaped rise and fall of pitch, timing, and loudness that a listener hears as a single bodily-derived movement, such as a reaching or a settling, within a phrase. Hatten's 2004 Interpreting Musical Gestures, Topics, and Tropes set out a systematic theory of these units, treating them as meaningful shapes that a performer projects and a listener follows in works by Mozart, Beethoven, and Schubert. The referent here is a heard motion in sound, described in terms of a musical phrase, not a hand movement made while talking, an input read by a machine, or a signal produced by an ape.[7]

Unit

One expressive shape of motion within a musical phrase

Recovery means

Not applicable

Measured by

Analysed from the score and from performance timing, dynamics, and pitch contour