  Heterogeneous Federated Learning (HtFL) enables collaborative learning on
multiple clients with different model architectures while preserving privacy.
Despite recent research progress, knowledge sharing in HtFL is still difficult
due to data and model heterogeneity. To tackle this issue, we leverage the
knowledge stored in pre-trained generators and propose a new upload-efficient
knowledge transfer scheme called Federated Knowledge-Transfer Loop (FedKTL).
Our FedKTL can produce client-task-related prototypical image-vector pairs via
the generator's inference on the server. With these pairs, each client can
transfer pre-existing knowledge from the generator to its local model through
an additional supervised local task. We conduct extensive experiments on four
datasets under two types of data heterogeneity with 14 kinds of models
including CNNs and ViTs. Results show that our upload-efficient FedKTL
surpasses seven state-of-the-art methods by up to 7.31% in accuracy. Moreover,
our knowledge transfer scheme is applicable in scenarios with only one edge
client. Code: https://github.com/TsingZ0/FedKTL
