Construct · 1 fields · 1 citations

induction heads: a standalone construct

One name. One construct. A landmark on the map.

● anchor · a standalone construct Nearest construct by meaning·not indexed
inductionheads1 CONSTRUCTInduction headsOLSSON · EXPLAINABLE

A single standalone construct; nearby constructs are located against it in the Collider. Tap a field for its definition, unit, and measures.

Explainable artificial intelligence

attention-head circuits that find an earlier occurrence of the current token and copy what followed it, proposed as a mechanism behind in-context learning.

In explainable artificial intelligence, induction heads refers to attention-head circuits that find an earlier occurrence of the current token and copy what followed it, proposed as a mechanism behind in-context learning. This meaning comes from Olsson (2022) in arXiv.[1]

Unit
Recovery means
Measured by

Next

Run it through the Collider, or take the citations with you.

See how induction heads sits against neighbouring constructs in the Collider, or export every reference on this page in one click.

Open in ColliderExport BibTeXReport this entry

Nearby constructs

current3 fields · 6 citationsjingle · one word, three constructsexplainable artificial intelligence1 fields · 1 citationsanchor · a standalone constructin-context learning1 fields · 1 citationsanchor · a standalone construct

References

  1. Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., & Olah, C. (2022). In-context Learning and Induction Heads. arXiv. https://doi.org/10.48550/arxiv.2209.11895

Cited on this page?

Confirm your construct, or correct us.

If your work is cited behind one of these entries, you know better than we do whether we captured the meaning right. Confirm it, suggest an edit, or dispute it, and your name goes on the entry as an author who has checked it, linked to your ORCID.

Claim your construct