  Federated Learning (FL) is an emerging distributed machine learning approach
that preserves client privacy by storing data on edge devices. However, data
heterogeneity among clients presents challenges in training models that perform
well on all local distributions. Recent studies have proposed clustering as a
solution to tackle client heterogeneity in FL by grouping clients with
distribution shifts into different clusters. However, the diverse learning
frameworks used in current clustered FL methods make it challenging to
integrate various clustered FL methods, gather their benefits, and make further
improvements.
  To this end, this paper presents a comprehensive investigation into current
clustered FL methods and proposes a four-tier framework, namely HCFL, to
encompass and extend existing approaches. Based on the HCFL, we identify the
remaining challenges associated with current clustering methods in each tier
and propose an enhanced clustering method called HCFL+ to address these
challenges. Through extensive numerical evaluations, we showcase the
effectiveness of our clustering framework and the improved components. Our code
will be publicly available.
