  In the field of heterogeneous federated learning (FL), the key challenge is
to efficiently and collaboratively train models across multiple clients with
different data distributions, model structures, task objectives, computational
capabilities, and communication resources. This diversity leads to significant
heterogeneity, which increases the complexity of model training. In this paper,
we first outline the basic concepts of heterogeneous federated learning and
summarize the research challenges in federated learning in terms of five
aspects: data, model, task, device, and communication. In addition, we explore
how existing state-of-the-art approaches cope with the heterogeneity of
federated learning, and categorize and review these approaches at three
different levels: data-level, model-level, and architecture-level.
Subsequently, the paper extensively discusses privacy-preserving strategies in
heterogeneous federated learning environments. Finally, the paper discusses
current open issues and directions for future research, aiming to promote the
further development of heterogeneous federated learning.
