  The increasing concerns regarding the privacy of machine learning models have
catalyzed the exploration of machine unlearning, i.e., a process that removes
the influence of training data on machine learning models. This concern also
arises in the realm of federated learning, prompting researchers to address the
federated unlearning problem. However, federated unlearning remains
challenging. Existing unlearning methods can be broadly categorized into two
approaches, i.e., exact unlearning and approximate unlearning. Firstly,
implementing exact unlearning, which typically relies on the
partition-aggregation framework, in a distributed manner does not improve time
efficiency theoretically. Secondly, existing federated (approximate) unlearning
methods suffer from imprecise data influence estimation, significant
computational burden, or both. To this end, we propose a novel federated
unlearning framework based on incremental learning, which is independent of
specific models and federated settings. Our framework differs from existing
federated unlearning methods that rely on approximate retraining or data
influence estimation. Instead, we leverage new memories to overwrite old ones,
imitating the process of \textit{active forgetting} in neurology. Specifically,
the model, intended to unlearn, serves as a student model that continuously
learns from randomly initiated teacher models. To preserve catastrophic
forgetting of non-target data, we utilize elastic weight consolidation to
elastically constrain weight change. Extensive experiments on three benchmark
datasets demonstrate the efficiency and effectiveness of our proposed method.
The result of backdoor attacks demonstrates that our proposed method achieves
satisfying completeness.
