  Recently, federated learning (FL) is popular for its privacy-preserving and
collaborative learning abilities. However, under statistically heterogeneous
scenarios, we observe that biased data domains on clients cause a
representation bias phenomenon and further degenerate generic representations
during local training, i.e., the representation degeneration phenomenon. To
address these issues, we propose a general framework Domain Bias Eliminator
(DBE) for FL. Our theoretical analysis reveals that DBE can promote
bi-directional knowledge transfer between server and client, as it reduces the
domain discrepancy between server and client in representation space. Besides,
extensive experiments on four datasets show that DBE can greatly improve
existing FL methods in both generalization and personalization abilities. The
DBE-equipped FL method can outperform ten state-of-the-art personalized FL
methods by a large margin. Our code is public at
https://github.com/TsingZ0/DBE.
