  Existing approaches in Federated Learning (FL) mainly focus on sending model
parameters or gradients from clients to a server. However, these methods are
plagued by significant inefficiency, privacy, and security concerns. Thanks to
the emerging foundation generative models, we propose a novel federated
learning framework, namely Federated Generative Learning. In this framework,
each client can create text embeddings that are tailored to their local data,
and send embeddings to the server. Then the informative training data can be
synthesized remotely on the server using foundation generative models with
these embeddings, which can benefit FL tasks. Our proposed framework offers
several advantages, including increased communication efficiency, robustness to
data heterogeneity, substantial performance improvements, and enhanced privacy
protection. We validate these benefits through extensive experiments conducted
on 12 datasets. For example, on the ImageNet100 dataset with a highly skewed
data distribution, our method outperforms FedAvg by 12% in a single
communication round, compared to FedAvg's performance over 200 communication
rounds. We have released the code for all experiments conducted in this study.
