  Federated learning is promising for its capability to collaboratively train
models with multiple clients without accessing their data, but vulnerable when
clients' data distributions diverge from each other. This divergence further
leads to a dilemma: "Should we prioritize the learned model's generic
performance (for future use at the server) or its personalized performance (for
each client)?" These two, seemingly competing goals have divided the community
to focus on one or the other, yet in this paper we show that it is possible to
approach both at the same time. Concretely, we propose a novel federated
learning framework that explicitly decouples a model's dual duties with two
prediction tasks. On the one hand, we introduce a family of losses that are
robust to non-identical class distributions, enabling clients to train a
generic predictor with a consistent objective across them. On the other hand,
we formulate the personalized predictor as a lightweight adaptive module that
is learned to minimize each client's empirical risk on top of the generic
predictor. With this two-loss, two-predictor framework which we name Federated
Robust Decoupling (Fed-RoD), the learned model can simultaneously achieve
state-of-the-art generic and personalized performance, essentially bridging the
two tasks.
