  Federated learning (FL) is a new distributed machine learning framework that
can achieve reliably collaborative training without collecting users' private
data. However, due to FL's frequent communication and average aggregation
strategy, they experience challenges scaling to statistical diversity data and
large-scale models. In this paper, we propose a personalized FL framework,
named Tensor Decomposition based Personalized Federated learning (TDPFed), in
which we design a novel tensorized local model with tensorized linear layers
and convolutional layers to reduce the communication cost. TDPFed uses a
bi-level loss function to decouple personalized model optimization from the
global model learning by controlling the gap between the personalized model and
the tensorized local model. Moreover, an effective distributed learning
strategy and two different model aggregation strategies are well designed for
the proposed TDPFed framework. Theoretical convergence analysis and thorough
experiments demonstrate that our proposed TDPFed framework achieves
state-of-the-art performance while reducing the communication cost.
