  Federated Learning (FL) is a decentralized learning method used to train
machine learning algorithms. In FL, a global model iteratively collects the
parameters of local models without accessing their local data. However, a
significant challenge in FL is handling the heterogeneity of local data
distribution, which often results in a drifted global model that is difficult
to converge. To address this issue, current methods employ different strategies
such as knowledge distillation, weighted model aggregation, and multi-task
learning. These approaches are referred to as asynchronous FL, as they align
user models either locally or post-hoc, where model drift has already occurred
or has been underestimated. In this paper, we propose an active and synchronous
correlation approach to address the challenge of user heterogeneity in FL.
Specifically, our approach aims to approximate FL as standard deep learning by
actively and synchronously scheduling user learning pace in each round with a
dynamic multi-phase curriculum. A global curriculum is formed by an
auto-regressive auto-encoder that integrates all user curricula on the server.
This global curriculum is then divided into multiple phases and broadcast to
users to measure and align the domain-agnostic learning pace. Empirical studies
demonstrate that our approach outperforms existing asynchronous approaches in
terms of generalization performance, even in the presence of severe user
heterogeneity.
