Construct · 1 fields · 1 citations

dying ReLU: a standalone construct

One name. One construct. A landmark on the map.

● anchor · a standalone construct Nearest construct by meaning·not indexed
dying ReLU1 CONSTRUCTDying ReLULU · DEEP LEARNING

A single standalone construct; nearby constructs are located against it in the Collider. Tap a field for its definition, unit, and measures.

Deep learning

the failure in which rectified linear units become stuck outputting zero for all inputs, so they stop passing gradients and stop learning.

In deep learning, dying ReLU refers to the failure in which rectified linear units become stuck outputting zero for all inputs, so they stop passing gradients and stop learning. This meaning comes from Lu (2020) in Communications in Computational Physics.[1]

Unit
Recovery means
Measured by

Next

Run it through the Collider, or take the citations with you.

See how dying ReLU sits against neighbouring constructs in the Collider, or export every reference on this page in one click.

Open in ColliderExport BibTeXReport this entry

Nearby constructs

deep learning1 fields · 1 citationsanchor · a standalone constructfailure4 fields · 6 citationsjingle · one word, four constructslearning6 fields · 14 citationsjingle · one name, six constructs

References

  1. Lu, L., Shin, Y., Su, Y., & Em Karniadakis, G. (2020). Dying ReLU and Initialization: Theory and Numerical Examples. Communications in Computational Physics, 28(5), 1671-1706. https://doi.org/10.4208/cicp.oa-2020-0165

Cited on this page?

Confirm your construct, or correct us.

If your work is cited behind one of these entries, you know better than we do whether we captured the meaning right. Confirm it, suggest an edit, or dispute it, and your name goes on the entry as an author who has checked it, linked to your ORCID.

Claim your construct