Construct · 1 fields · 1 citations

anisotropy of contextual embeddings: a standalone construct

One name. One construct. A landmark on the map.

● anchor · a standalone construct Nearest construct by meaning·not indexed
anisotropy ofcontextual embeddings1 CONSTRUCTAnisotropy of contextualETHAYARAJH · NATURAL LANGUAGE

A single standalone construct; nearby constructs are located against it in the Collider. Tap a field for its definition, unit, and measures.

Natural language processing

the tendency of contextual word vectors to crowd into a narrow cone of the space, so even unrelated words appear highly similar.

In natural language processing, anisotropy of contextual embeddings refers to the tendency of contextual word vectors to crowd into a narrow cone of the space, so even unrelated words appear highly similar. This meaning comes from Ethayarajh (2019) in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).[1]

Unit
Recovery means
Measured by

Next

Run it through the Collider, or take the citations with you.

See how anisotropy of contextual embeddings sits against neighbouring constructs in the Collider, or export every reference on this page in one click.

Open in ColliderExport BibTeXReport this entry

Nearby constructs

anisotropy2 fields · 2 citationsinvariant · one idea, two scales: direction-dependence in crystal and in cosmosjoint2 fields · 2 citationsjingle · one word, two constructsnatural3 fields · 7 citationsjingle · one word, three constructs

References

  1. Ethayarajh, K. (2019). How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). https://doi.org/10.18653/v1/d19-1006

Cited on this page?

Confirm your construct, or correct us.

If your work is cited behind one of these entries, you know better than we do whether we captured the meaning right. Confirm it, suggest an edit, or dispute it, and your name goes on the entry as an author who has checked it, linked to your ORCID.

Claim your construct