  Federated Learning (FL) has recently made significant progress as a new
machine learning paradigm for privacy protection. Due to the high communication
cost of traditional FL, one-shot federated learning is gaining popularity as a
way to reduce communication cost between clients and the server. Most of the
existing one-shot FL methods are based on Knowledge Distillation; however,
{distillation based approach requires an extra training phase and depends on
publicly available data sets or generated pseudo samples.} In this work, we
consider a novel and challenging cross-silo setting: performing a single round
of parameter aggregation on the local models without server-side training. In
this setting, we propose an effective algorithm for Model Aggregation via
Exploring Common Harmonized Optima (MA-Echo), which iteratively updates the
parameters of all local models to bring them close to a common low-loss area on
the loss surface, without harming performance on their own data sets at the
same time. Compared to the existing methods, MA-Echo can work well even in
extremely non-identical data distribution settings where the support categories
of each local model have no overlapped labels with those of the others. We
conduct extensive experiments on two popular image classification data sets to
compare the proposed method with existing methods and demonstrate the
effectiveness of MA-Echo, which clearly outperforms the state-of-the-arts. The
source code can be accessed in \url{https://github.com/FudanVI/MAEcho}.
