  Federated Learning (FL) is an emerging distributed learning paradigm under
privacy constraint. Data heterogeneity is one of the main challenges in FL,
which results in slow convergence and degraded performance. Most existing
approaches only tackle the heterogeneity challenge by restricting the local
model update in client, ignoring the performance drop caused by direct global
model aggregation. Instead, we propose a data-free knowledge distillation
method to fine-tune the global model in the server (FedFTG), which relieves the
issue of direct model aggregation. Concretely, FedFTG explores the input space
of local models through a generator, and uses it to transfer the knowledge from
local models to the global model. Besides, we propose a hard sample mining
scheme to achieve effective knowledge distillation throughout the training. In
addition, we develop customized label sampling and class-level ensemble to
derive maximum utilization of knowledge, which implicitly mitigates the
distribution discrepancy across clients. Extensive experiments show that our
FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and
can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and
SCAFFOLD.
