  Personalized Federated Learning (pFL) has emerged as a promising solution to
tackle data heterogeneity across clients in FL. However, existing pFL methods
either (1) introduce high communication and computation costs or (2) overfit to
local data, which can be limited in scope, and are vulnerable to evolved test
samples with natural shifts. In this paper, we propose PerAda, a
parameter-efficient pFL framework that reduces communication and computational
costs and exhibits superior generalization performance, especially under
test-time distribution shifts. PerAda reduces the costs by leveraging the power
of pretrained models and only updates and communicates a small number of
additional parameters from adapters. PerAda has good generalization since it
regularizes each client's personalized adapter with a global adapter, while the
global adapter uses knowledge distillation to aggregate generalized information
from all clients. Theoretically, we provide generalization bounds to explain
why PerAda improves generalization, and we prove its convergence to stationary
points under non-convex settings. Empirically, PerAda demonstrates competitive
personalized performance (+4.85% on CheXpert) and enables better
out-of-distribution generalization (+5.23% on CIFAR-10-C) on different datasets
across natural and medical domains compared with baselines, while only updating
12.6% of parameters per model based on the adapter. Our code is available at
https://github.com/NVlabs/PerAda.
