  Federated Learning (FL) models often experience client drift caused by
heterogeneous data, where the distribution of data differs across clients. To
address this issue, advanced research primarily focuses on manipulating the
existing gradients to achieve more consistent client models. In this paper, we
present an alternative perspective on client drift and aim to mitigate it by
generating improved local models. First, we analyze the generalization
contribution of local training and conclude that this generalization
contribution is bounded by the conditional Wasserstein distance between the
data distribution of different clients. Then, we propose FedImpro, to construct
similar conditional distributions for local training. Specifically, FedImpro
decouples the model into high-level and low-level components, and trains the
high-level portion on reconstructed feature distributions. This approach
enhances the generalization contribution and reduces the dissimilarity of
gradients in FL. Experimental results show that FedImpro can help FL defend
against data heterogeneity and enhance the generalization performance of the
model.
