  Multi-server Federated learning (FL) has been considered as a promising
solution to address the limited communication resource problem of single-server
FL. We consider a typical multi-server FL architecture, where the coverage
areas of regional servers may overlap. The key point of this architecture is
that the clients located in the overlapping areas update their local models
based on the average model of all accessible regional models, which enables
indirect model sharing among different regional servers. Due to the complicated
network topology, the convergence analysis is much more challenging than
single-server FL. In this paper, we firstly propose a novel MS-FedAvg algorithm
for this multi-server FL architecture and analyze its convergence on non-iid
datasets for general non-convex settings. Since the number of clients located
in each regional server is much less than in single-server FL, the bandwidth of
each client should be large enough to successfully communicate training models
with the server, which indicates that full client participation can work in
multi-server FL. Also, we provide the convergence analysis of the partial
client participation scheme and develop a new biased partial participation
strategy to further accelerate convergence. Our results indicate that the
convergence results highly depend on the ratio of the number of clients in each
area type to the total number of clients in all three strategies. The extensive
experiments show remarkable performance and support our theoretical results.
