  The federated learning (FL) framework enables edge clients to collaboratively
learn a shared inference model while keeping privacy of training data on
clients. Recently, many heuristics efforts have been made to generalize
centralized adaptive optimization methods, such as SGDM, Adam, AdaGrad, etc.,
to federated settings for improving convergence and accuracy. However, there is
still a paucity of theoretical principles on where to and how to design and
utilize adaptive optimization methods in federated settings. This work aims to
develop novel adaptive optimization methods for FL from the perspective of
dynamics of ordinary differential equations (ODEs). First, an analytic
framework is established to build a connection between federated optimization
methods and decompositions of ODEs of corresponding centralized optimizers.
Second, based on this analytic framework, a momentum decoupling adaptive
optimization method, FedDA, is developed to fully utilize the global momentum
on each local iteration and accelerate the training convergence. Last but not
least, full batch gradients are utilized to mimic centralized optimization in
the end of the training process to ensure the convergence and overcome the
possible inconsistency caused by adaptive optimization methods.
