  Federated learning (FL) has emerged as a prominent approach for collaborative
training of machine learning models across distributed clients while preserving
data privacy. However, the quest to balance acceleration and stability becomes
a significant challenge in FL, especially on the client-side. In this paper, we
introduce FedCAda, an innovative federated client adaptive algorithm designed
to tackle this challenge. FedCAda leverages the Adam algorithm to adjust the
correction process of the first moment estimate $m$ and the second moment
estimate $v$ on the client-side and aggregate adaptive algorithm parameters on
the server-side, aiming to accelerate convergence speed and communication
efficiency while ensuring stability and performance. Additionally, we
investigate several algorithms incorporating different adjustment functions.
This comparative analysis revealed that due to the limited information
contained within client models from other clients during the initial stages of
federated learning, more substantial constraints need to be imposed on the
parameters of the adaptive algorithm. As federated learning progresses and
clients gather more global information, FedCAda gradually diminishes the impact
on adaptive parameters. These findings provide insights for enhancing the
robustness and efficiency of algorithmic improvements. Through extensive
experiments on computer vision (CV) and natural language processing (NLP)
datasets, we demonstrate that FedCAda outperforms the state-of-the-art methods
in terms of adaptability, convergence, stability, and overall performance. This
work contributes to adaptive algorithms for federated learning, encouraging
further exploration.
