  Addressing the issues of hallucinations and outdated knowledge in large
language models is critical for their reliable application. Model Editing
presents a promising avenue for mitigating these challenges in a cost-effective
manner. However, existing methods often suffer from unsatisfactory
generalization and unintended effects on non-edited samples. To overcome these
limitations, we introduce a novel approach: Scalable Model Editing via
Customized Expert Networks (SCEN), which is a two-stage continuous training
paradigm. Specifically, in the first stage, we train lightweight expert
networks individually for each piece of knowledge that needs to be updated.
Subsequently, we train a corresponding indexing neuron for each expert to
control the activation state of that expert. We conducted a series of
experiments on the ZsRE and Hallucination benchmarks by tuning the advanced
open-source LLM, Llama2, achieving state-of-the-art results compared to current
mainstream methods. Our code is available at
https://github.com/TAL-auroraX/SCEN.
