  Recently, while large language models (LLMs) have demonstrated impressive
results, they still suffer from hallucination, i.e., the generation of false
information. Model editing is the task of fixing factual mistakes in LLMs; yet,
most previous works treat it as a one-time task, paying little attention to
ever-emerging mistakes generated by LLMs. We address the task of sequential
model editing (SME) that aims to rectify mistakes continuously. A Dynamic
Auxiliary Fusion Network (DAFNet) is designed to enhance the semantic
interaction among the factual knowledge within the entire sequence, preventing
catastrophic forgetting during the editing process of multiple knowledge
triples. Specifically, (1) for semantic fusion within a relation triple, we
aggregate the intra-editing attention flow into auto-regressive self-attention
with token-level granularity in LLMs. We further leverage multi-layer diagonal
inter-editing attention flow to update the weighted representations of the
entire sequence-level granularity. (2) Considering that auxiliary parameters
are required to store the knowledge for sequential editing, we construct a new
dataset named \textbf{DAFSet}, fulfilling recent, popular, long-tail and robust
properties to enhance the generality of sequential editing. Experiments show
DAFNet significantly outperforms strong baselines in single-turn and sequential
editing. The usage of DAFSet also consistently improves the performance of
other auxiliary network-based methods in various scenarios
