  Reducing communication overhead in federated learning (FL) is challenging but
crucial for large-scale distributed privacy-preserving machine learning. While
methods utilizing sparsification or others can largely lower the communication
overhead, the convergence rate is also greatly compromised. In this paper, we
propose a novel method, named single-step synthetic features compressor (3SFC),
to achieve communication-efficient FL by directly constructing a tiny synthetic
dataset based on raw gradients. Thus, 3SFC can achieve an extremely low
compression rate when the constructed dataset contains only one data sample.
Moreover, 3SFC's compressing phase utilizes a similarity-based objective
function so that it can be optimized with just one step, thereby considerably
improving its performance and robustness. In addition, to minimize the
compressing error, error feedback (EF) is also incorporated into 3SFC.
Experiments on multiple datasets and models suggest that 3SFC owns
significantly better convergence rates compared to competing methods with lower
compression rates (up to 0.02%). Furthermore, ablation studies and
visualizations show that 3SFC can carry more information than competing methods
for every communication round, further validating its effectiveness.
