  Recently, Heterogeneous Federated Learning (HtFL) has attracted attention due
to its ability to support heterogeneous models and data. To reduce the high
communication cost of transmitting model parameters, a major challenge in HtFL,
prototype-based HtFL methods are proposed to solely share class
representatives, a.k.a, prototypes, among heterogeneous clients while
maintaining the privacy of clients' models. However, these prototypes are
naively aggregated into global prototypes on the server using weighted
averaging, resulting in suboptimal global knowledge which negatively impacts
the performance of clients. To overcome this challenge, we introduce a novel
HtFL approach called FedTGP, which leverages our Adaptive-margin-enhanced
Contrastive Learning (ACL) to learn Trainable Global Prototypes (TGP) on the
server. By incorporating ACL, our approach enhances prototype separability
while preserving semantic meaning. Extensive experiments with twelve
heterogeneous models demonstrate that our FedTGP surpasses state-of-the-art
methods by up to 9.08% in accuracy while maintaining the communication and
privacy advantages of prototype-based HtFL. Our code is available at
https://github.com/TsingZ0/FedTGP.
