  Federated learning has gained significant attention due to its groundbreaking
ability to enable distributed learning while maintaining privacy constraints.
However, as a consequence of data heterogeneity among decentralized devices, it
inherently experiences significant learning degradation and slow convergence
speed. Therefore, it is natural to employ the concept of clustering homogeneous
clients into the same group, allowing only the model weights within each group
to be aggregated. While most existing clustered federated learning methods
employ either model gradients or inference outputs as metrics for client
partitioning, with the goal of grouping similar devices together, may still
have heterogeneity within each cluster. Moreover, there is a scarcity of
research exploring the underlying reasons for determining the appropriate
timing for clustering, resulting in the common practice of assigning each
client to its own individual cluster, particularly in the context of highly non
independent and identically distributed (Non-IID) data. In this paper, we
introduce a two-stage decoupling federated learning algorithm with adaptive
personalization layers named FedTSDP, where client clustering is performed
twice according to inference outputs and model weights, respectively. Hopkins
amended sampling is adopted to determine the appropriate timing for clustering
and the sampling weight of public unlabeled data. In addition, a simple yet
effective approach is developed to adaptively adjust the personalization layers
based on varying degrees of data skew. Experimental results show that our
proposed method has reliable performance on both IID and non-IID scenarios.
