  As a promising distributed machine learning paradigm, Federated Learning (FL)
has attracted increasing attention to deal with data silo problems without
compromising user privacy. By adopting the classic one-to-multi training scheme
(i.e., FedAvg), where the cloud server dispatches one single global model to
multiple involved clients, conventional FL methods can achieve collaborative
model training without data sharing. However, since only one global model
cannot always accommodate all the incompatible convergence directions of local
models, existing FL approaches greatly suffer from inferior classification
accuracy. To address this issue, we present an efficient FL framework named
FedCross, which uses a novel multi-to-multi FL training scheme based on our
proposed multi-model cross-aggregation approach. Unlike traditional FL methods,
in each round of FL training, FedCross uses multiple middleware models to
conduct weighted fusion individually. Since the middleware models used by
FedCross can quickly converge into the same flat valley in terms of loss
landscapes, the generated global model can achieve a well-generalization.
Experimental results on various well-known datasets show that, compared with
state-of-the-art FL methods, FedCross can significantly improve FL accuracy
within both IID and non-IID scenarios without causing additional communication
overhead.
