  Federated learning (FL) on heterogeneous data (non-IID data) has recently
received great attention. Most existing methods focus on studying the
convergence guarantees for the global objective. While these methods can
guarantee the decrease of the global objective in each communication round,
they fail to ensure risk decrease for each client. In this paper, to address
the problem,we propose FedCOME, which introduces a consensus mechanism to
enforce decreased risk for each client after each training round. In
particular, we allow a slight adjustment to a client's gradient on the server
side, which generates an acute angle between the corrected gradient and the
original ones of other clients. We theoretically show that the consensus
mechanism can guarantee the convergence of the global objective. To generalize
the consensus mechanism to the partial participation FL scenario, we devise a
novel client sampling strategy to select the most representative clients for
the global data distribution. Training on these selected clients with the
consensus mechanism could empirically lead to risk decrease for clients that
are not selected. Finally, we conduct extensive experiments on four benchmark
datasets to show the superiority of FedCOME against other state-of-the-art
methods in terms of effectiveness, efficiency and fairness. For
reproducibility, we make our source code publicly available at:
\url{https://github.com/fedcome/fedcome}.
