  Federated Learning (FL) is a way for machines to learn from data that is kept
locally, in order to protect the privacy of clients. This is typically done
using local SGD, which helps to improve communication efficiency. However, such
a scheme is currently constrained by slow and unstable convergence due to the
variety of data on different clients' devices. In this work, we identify three
under-explored phenomena of biased local learning that may explain these
challenges caused by local updates in supervised FL. As a remedy, we propose
FedBR, a novel unified algorithm that reduces the local learning bias on
features and classifiers to tackle these challenges. FedBR has two components.
The first component helps to reduce bias in local classifiers by balancing the
output of the models. The second component helps to learn local features that
are similar to global features, but different from those learned from other
data sources. We conducted several experiments to test \algopt and found that
it consistently outperforms other SOTA FL methods. Both of its components also
individually show performance gains. Our code is available at
https://github.com/lins-lab/fedbr.
