  Large language models (LLMs) store extensive factual knowledge, but the
underlying mechanisms remain unclear. Previous research suggests that factual
knowledge is stored within multi-layer perceptron weights, and some storage
units exhibit degeneracy, referred to as Degenerate Knowledge Neurons (DKNs).
Despite the novelty and unique properties of this concept, it has not been
rigorously defined or systematically studied. We first consider the connection
weight patterns of MLP neurons and define DKNs from both structural and
functional aspects. Based on this, we introduce the Neurological Topology
Clustering method, which allows the formation of DKNs in any numbers and
structures, leading to a more accurate DKN acquisition. Furthermore, inspired
by cognitive science, we explore the relationship between DKNs and the
robustness, evolvability, and complexity of LLMs. Our execution of 34
experiments under 6 settings demonstrates the connection between DKNs and these
three properties. The code will be available soon.
