  In contrast to centralized model training that involves data collection,
federated learning (FL) enables remote clients to collaboratively train a model
without exposing their private data. However, model performance usually
degrades in FL due to the heterogeneous data generated by clients of diverse
characteristics. One promising strategy to maintain good performance is by
limiting the local training from drifting far away from the global model.
Previous studies accomplish this by regularizing the distance between the
representations learned by the local and global models. However, they only
consider representations from the early layers of a model or the layer
preceding the output layer. In this study, we introduce FedIntR, which provides
a more fine-grained regularization by integrating the representations of
intermediate layers into the local training process. Specifically, FedIntR
computes a regularization term that encourages the closeness between the
intermediate layer representations of the local and global models.
Additionally, FedIntR automatically determines the contribution of each layer's
representation to the regularization term based on the similarity between local
and global representations. We conduct extensive experiments on various
datasets to show that FedIntR can achieve equivalent or higher performance
compared to the state-of-the-art approaches. Our code is available at
https://github.com/YLTun/FedIntR.
