  Federated learning (FL) enables multiple clients to train a machine learning
model collaboratively without exchanging their local data. Federated unlearning
is an inverse FL process that aims to remove a specified target client's
contribution in FL to satisfy the user's right to be forgotten. Most existing
federated unlearning algorithms require the server to store the history of the
parameter updates, which is not applicable in scenarios where the server
storage resource is constrained. In this paper, we propose a
simple-yet-effective subspace based federated unlearning method, dubbed SFU,
that lets the global model perform gradient ascent in the orthogonal space of
input gradient spaces formed by other clients to eliminate the target client's
contribution without requiring additional storage. Specifically, the server
first collects the gradients generated from the target client after performing
gradient ascent, and the input representation matrix is computed locally by the
remaining clients. We also design a differential privacy method to protect the
privacy of the representation matrix. Then the server merges those
representation matrices to get the input gradient subspace and updates the
global model in the orthogonal subspace of the input gradient subspace to
complete the forgetting task with minimal model performance degradation.
Experiments on MNIST, CIFAR10, and CIFAR100 show that SFU outperforms several
state-of-the-art (SOTA) federated unlearning algorithms by a large margin in
various settings.
