  Federated Learning (FL) aims to train machine learning models for multiple
clients without sharing their own private data. Due to the heterogeneity of
clients' local data distribution, recent studies explore the personalized FL
that learns and deploys distinct local models with the help of auxiliary global
models. However, the clients can be heterogeneous in terms of not only local
data distribution, but also their computation and communication resources. The
capacity and efficiency of personalized models are restricted by the
lowest-resource clients, leading to sub-optimal performance and limited
practicality of personalized FL. To overcome these challenges, we propose a
novel approach named pFedGate for efficient personalized FL by adaptively and
efficiently learning sparse local models. With a lightweight trainable gating
layer, pFedGate enables clients to reach their full potential in model capacity
by generating different sparse models accounting for both the heterogeneous
data distributions and resource constraints. Meanwhile, the computation and
communication efficiency are both improved thanks to the adaptability between
the model sparsity and clients' resources. Further, we theoretically show that
the proposed pFedGate has superior complexity with guaranteed convergence and
generalization error. Extensive experiments show that pFedGate achieves
superior global accuracy, individual accuracy and efficiency simultaneously
over state-of-the-art methods. We also demonstrate that pFedGate performs
better than competitors in the novel clients participation and partial clients
participation scenarios, and can learn meaningful sparse local models adapted
to different data distributions.
