  Named Entity Recognition (NER) models capable of Continual Learning (CL) are
realistically valuable in areas where entity types continuously increase (e.g.,
personal assistants). Meanwhile the learning paradigm of NER advances to new
patterns such as the span-based methods. However, its potential to CL has not
been fully explored. In this paper, we propose SpanKL, a simple yet effective
Span-based model with Knowledge distillation (KD) to preserve memories and
multi-Label prediction to prevent conflicts in CL-NER. Unlike prior sequence
labeling approaches, the inherently independent modeling in span and entity
level with the designed coherent optimization on SpanKL promotes its learning
at each incremental step and mitigates the forgetting. Experiments on synthetic
CL datasets derived from OntoNotes and Few-NERD show that SpanKL significantly
outperforms previous SoTA in many aspects, and obtains the smallest gap from CL
to the upper bound revealing its high practiced value. The code is available at
https://github.com/Qznan/SpanKL.
