  Traditional machine learning relies on a centralized data pipeline, i.e.,
data are provided to a central server for model training. In many applications,
however, data are inherently fragmented. Such a decentralized nature of these
databases presents the biggest challenge for collaboration: sending all
decentralized datasets to a central server raises serious privacy concerns.
Although there has been a joint effort in tackling such a critical issue by
proposing privacy-preserving machine learning frameworks, such as federated
learning, most state-of-the-art frameworks are built still in a centralized
way, in which a central client is needed for collecting and distributing model
information (instead of data itself) from every other client, leading to high
communication pressure and high vulnerability when there exists a failure at or
attack on the central client. Here we propose a principled decentralized
federated learning algorithm (DeceFL), which does not require a central client
and relies only on local information transmission between clients and their
neighbors, representing a fully decentralized learning framework. It has been
further proven that every client reaches the global minimum with zero
performance gap and achieves the same convergence rate $O(1/T)$ (where $T$ is
the number of iterations in gradient descent) as centralized federated learning
when the loss function is smooth and strongly convex. Finally, the proposed
algorithm has been applied to a number of applications to illustrate its
effectiveness for both convex and nonconvex loss functions, demonstrating its
applicability to a wide range of real-world medical and industrial
applications.
