  Federated Learning (FL) offers a collaborative training framework, allowing
multiple clients to contribute to a shared model without compromising data
privacy. Due to the heterogeneous nature of local datasets, updated client
models may overfit and diverge from one another, commonly known as the problem
of client drift. In this paper, we propose FedBug (Federated Learning with
Bottom-Up Gradual Unfreezing), a novel FL framework designed to effectively
mitigate client drift. FedBug adaptively leverages the client model parameters,
distributed by the server at each global round, as the reference points for
cross-client alignment. Specifically, on the client side, FedBug begins by
freezing the entire model, then gradually unfreezes the layers, from the input
layer to the output layer. This bottom-up approach allows models to train the
newly thawed layers to project data into a latent space, wherein the separating
hyperplanes remain consistent across all clients. We theoretically analyze
FedBug in a novel over-parameterization FL setup, revealing its superior
convergence rate compared to FedAvg. Through comprehensive experiments,
spanning various datasets, training conditions, and network architectures, we
validate the efficacy of FedBug. Our contributions encompass a novel FL
framework, theoretical analysis, and empirical validation, demonstrating the
wide potential and applicability of FedBug.
