  Federated learning (FL), as a collaborative distributed training paradigm
with several edge computing devices under the coordination of a centralized
server, is plagued by inconsistent local stationary points due to the
heterogeneity of the local partial participation clients, which precipitates
the local client-drifts problems and sparks off the unstable and slow
convergence, especially on the aggravated heterogeneous dataset. To address
these issues, we propose a novel federated learning algorithm, named FedMIM,
which adopts the multi-step inertial momentum on the edge devices and enhances
the local consistency for free during the training to improve the robustness of
the heterogeneity. Specifically, we incorporate the weighted global gradient
estimations as the inertial correction terms to guide both the local iterates
and stochastic gradient estimation, which can reckon the global objective
optimization on the edges' heterogeneous dataset naturally and maintain the
demanding consistent iteration locally. Theoretically, we show that FedMIM
achieves the $\mathcal{O}(\frac{1}{\sqrt{SKT}})$ convergence rate with a linear
speedup property with respect to the number of selected clients $S$ and proper
local interval $K$ in communication round $T$ without convex assumption.
Empirically, we conduct comprehensive experiments on various real-world
datasets and demonstrate the efficacy of the proposed FedMIM against several
state-of-the-art baselines.
