  Federated Learning (FL) is a privacy-constrained decentralized machine
learning paradigm in which clients enable collaborative training without
compromising private data. However, how to learn a robust global model in the
data-heterogeneous and model-heterogeneous FL scenarios is challenging. To
address it, we resort to data-free knowledge distillation to propose a new FL
method (namely DFRD). DFRD equips a conditional generator on the server to
approximate the training space of the local models uploaded by clients, and
systematically investigates its training in terms of fidelity, transferability}
and diversity. To overcome the catastrophic forgetting of the global model
caused by the distribution shifts of the generator across communication rounds,
we maintain an exponential moving average copy of the generator on the server.
Additionally, we propose dynamic weighting and label sampling to accurately
extract knowledge from local models. Finally, our extensive experiments on
various image classification tasks illustrate that DFRD achieves significant
performance gains compared to SOTA baselines.
