  Statistical heterogeneity is a root cause of tension among accuracy,
fairness, and robustness of federated learning (FL), and is key in paving a
path forward. Personalized FL (PFL) is an approach that aims to reduce the
impact of statistical heterogeneity by developing personalized models for
individual users, while also inherently providing benefits in terms of fairness
and robustness. However, existing PFL frameworks focus on improving the
performance of personalized models while neglecting the global model. Moreover,
these frameworks achieve sublinear convergence rates and rely on strong
assumptions. In this paper, we propose FLAME, an optimization framework by
utilizing the alternating direction method of multipliers (ADMM) to train
personalized and global models. We propose a model selection strategy to
improve performance in situations where clients have different types of
heterogeneous data. Our theoretical analysis establishes the global convergence
and two kinds of convergence rates for FLAME under mild assumptions. We
theoretically demonstrate that FLAME is more robust and fair than the
state-of-the-art methods on a class of linear problems. Our experimental
findings show that FLAME outperforms state-of-the-art methods in convergence
and accuracy, and it achieves higher test accuracy under various attacks and
performs more uniformly across clients.
