  Federated learning is a distributed machine learning paradigm where multiple
data owners (clients) collaboratively train one machine learning model while
keeping data on their own devices. The heterogeneity of client datasets is one
of the most important challenges of federated learning algorithms. Studies have
found performance reduction with standard federated algorithms, such as FedAvg,
on non-IID data. Many existing works on handling non-IID data adopt the same
aggregation framework as FedAvg and focus on improving model updates either on
the server side or on clients. In this work, we tackle this challenge in a
different view by introducing redistribution rounds that delay the aggregation.
We perform experiments on multiple tasks and show that the proposed framework
significantly improves the performance on non-IID data.
