  Federated learning has been a hot research topic in enabling the
collaborative training of machine learning models among different organizations
under the privacy restrictions. As researchers try to support more machine
learning models with different privacy-preserving approaches, there is a
requirement in developing systems and infrastructures to ease the development
of various federated learning algorithms. Similar to deep learning systems such
as PyTorch and TensorFlow that boost the development of deep learning,
federated learning systems (FLSs) are equivalently important, and face
challenges from various aspects such as effectiveness, efficiency, and privacy.
In this survey, we conduct a comprehensive review on federated learning
systems. To achieve smooth flow and guide future research, we introduce the
definition of federated learning systems and analyze the system components.
Moreover, we provide a thorough categorization for federated learning systems
according to six different aspects, including data distribution, machine
learning model, privacy mechanism, communication architecture, scale of
federation and motivation of federation. The categorization can help the design
of federated learning systems as shown in our case studies. By systematically
summarizing the existing federated learning systems, we present the design
factors, case studies, and future research opportunities.
