  We formalize and study a phenomenon called feature collapse that makes
precise the intuitive idea that entities playing a similar role in a learning
task receive similar representations. As feature collapse requires a notion of
task, we leverage a simple but prototypical NLP task to study it. We start by
showing experimentally that feature collapse goes hand in hand with
generalization. We then prove that, in the large sample limit, distinct words
that play identical roles in this NLP task receive identical local feature
representations in a neural network. This analysis reveals the crucial role
that normalization mechanisms, such as LayerNorm, play in feature collapse and
in generalization.
