  Federated Learning (FL) is a machine learning paradigm that safeguards
privacy by retaining client data on edge devices. However, optimizing FL in
practice can be challenging due to the diverse and heterogeneous nature of the
learning system. Though recent research has focused on improving the
optimization of FL when distribution shifts occur among clients, ensuring
global performance when multiple types of distribution shifts occur
simultaneously among clients -- such as feature distribution shift, label
distribution shift, and concept shift -- remain under-explored. In this paper,
we identify the learning challenges posed by the simultaneous occurrence of
diverse distribution shifts and propose a clustering principle to overcome
these challenges. Through our research, we find that existing methods fail to
address the clustering principle. Therefore, we propose a novel clustering
algorithm framework, dubbed as FedRC, which adheres to our proposed clustering
principle by incorporating a bi-level optimization problem and a novel
objective function. Extensive experiments demonstrate that FedRC significantly
outperforms other SOTA cluster-based FL methods. Our code is available at
\url{https://github.com/LINs-lab/FedRC}.
