  Federated learning (FL) is a promising technique for addressing the rising
privacy and security issues. Its main ingredient is to cooperatively learn the
model among the distributed clients without uploading any sensitive data. In
this paper, we conducted a thorough review of the related works, following the
development context and deeply mining the key technologies behind FL from both
theoretical and practical perspectives. Specifically, we first classify the
existing works in FL architecture based on the network topology of FL systems
with detailed analysis and summarization. Next, we abstract the current
application problems, summarize the general techniques and frame the
application problems into the general paradigm of FL base models. Moreover, we
provide our proposed solutions for model training via FL. We have summarized
and analyzed the existing FedOpt algorithms, and deeply revealed the
algorithmic development principles of many first-order algorithms in depth,
proposing a more generalized algorithm design framework. Based on these
frameworks, we have instantiated FedOpt algorithms. As privacy and security is
the fundamental requirement in FL, we provide the existing attack scenarios and
the defense methods. To the best of our knowledge, we are among the first tier
to review the theoretical methodology and propose our strategies since there
are very few works surveying the theoretical approaches. Our survey targets
motivating the development of high-performance, privacy-preserving, and secure
methods to integrate FL into real-world applications.
