  Statistical heterogeneity severely limits the performance of federated
learning (FL), motivating several explorations e.g., FedProx, MOON and FedDyn,
to alleviate this problem. Despite effectiveness, their considered scenario
generally requires samples from almost all classes during the local training of
each client, although some covariate shifts may exist among clients. In fact,
the natural case of partially class-disjoint data (PCDD), where each client
contributes a few classes (instead of all classes) of samples, is practical yet
underexplored. Specifically, the unique collapse and invasion characteristics
of PCDD can induce the biased optimization direction in local training, which
prevents the efficiency of federated learning. To address this dilemma, we
propose a manifold reshaping approach called FedMR to calibrate the feature
space of local training. Our FedMR adds two interplaying losses to the vanilla
federated learning: one is intra-class loss to decorrelate feature dimensions
for anti-collapse; and the other one is inter-class loss to guarantee the
proper margin among categories in the feature expansion. We conduct extensive
experiments on a range of datasets to demonstrate that our FedMR achieves much
higher accuracy and better communication efficiency. Source code is available
at: https://github.com/MediaBrain-SJTU/FedMR.git.
