  Recently, emergence has received widespread attention from the research
community along with the success of large-scale models. Different from the
literature, we hypothesize a key factor that promotes the performance during
the increase of scale: the reduction of monosemantic neurons that can only form
one-to-one correlations with specific features. Monosemantic neurons tend to be
sparser and have negative impacts on the performance in large models. Inspired
by this insight, we propose an intuitive idea to identify monosemantic neurons
and inhibit them. However, achieving this goal is a non-trivial task as there
is no unified quantitative evaluation metric and simply banning monosemantic
neurons does not promote polysemanticity in neural networks. Therefore, we
first propose a new metric to measure the monosemanticity of neurons with the
guarantee of efficiency for online computation, then introduce a theoretically
supported method to suppress monosemantic neurons and proactively promote the
ratios of polysemantic neurons in training neural networks. We validate our
conjecture that monosemanticity brings about performance change at different
model scales on a variety of neural networks and benchmark datasets in
different areas, including language, image, and physics simulation tasks.
Further experiments validate our analysis and theory regarding the inhibition
of monosemanticity.
