  Continual Federated Learning (CFL) combines Federated Learning (FL), the
decentralized learning of a central model on a number of client devices that
may not communicate their data, and Continual Learning (CL), the learning of a
model from a continual stream of data without keeping the entire history. In
CL, the main challenge is \textit{forgetting} what was learned from past data.
While replay-based algorithms that keep a small pool of past training data are
effective to reduce forgetting, only simple replay sample selection strategies
have been applied to CFL in prior work, and no previous work has explored
coordination among clients for better sample selection. To bridge this gap, we
adapt a replay sample selection objective based on loss gradient diversity to
CFL and propose a new relaxation-based selection of samples to optimize the
objective. Next, we propose a practical algorithm to coordinate gradient-based
replay sample selection across clients without communicating private data. We
benchmark our coordinated and uncoordinated replay sample selection algorithms
against random sampling-based baselines with language models trained on a large
scale de-identified real-world text dataset. We show that gradient-based sample
selection methods both boost performance and reduce forgetting compared to
random sampling methods, with our coordination method showing gains early in
the low replay size regime (when the budget for storing past data is small).
