  Federated learning has received great attention for its capability to train a
large-scale model in a decentralized manner without needing to access user data
directly. It helps protect the users' private data from centralized collecting.
Unlike distributed machine learning, federated learning aims to tackle non-IID
data from heterogeneous sources in various real-world applications, such as
those on smartphones. Existing federated learning approaches usually adopt a
single global model to capture the shared knowledge of all users by aggregating
their gradients, regardless of the discrepancy between their data
distributions. However, due to the diverse nature of user behaviors, assigning
users' gradients to different global models (i.e., centers) can better capture
the heterogeneity of data distributions across users. Our paper proposes a
novel multi-center aggregation mechanism for federated learning, which learns
multiple global models from the non-IID user data and simultaneously derives
the optimal matching between users and centers. We formulate the problem as a
joint optimization that can be efficiently solved by a stochastic expectation
maximization (EM) algorithm. Our experimental results on benchmark datasets
show that our method outperforms several popular federated learning methods.
