  Federated learning (FL) has attracted growing attention since it allows for
privacy-preserving collaborative training on decentralized clients without
explicitly uploading sensitive data to the central server. However, recent
works have revealed that it still has the risk of exposing private data to
adversaries. In this paper, we conduct reconstruction attacks and enhance
inference attacks on various datasets to better understand that sharing trained
classification model parameters to a central server is the main problem of
privacy leakage in FL. To tackle this problem, a privacy-preserving image
distribution sharing scheme with GAN (PPIDSG) is proposed, which consists of a
block scrambling-based encryption algorithm, an image distribution sharing
method, and local classification training. Specifically, our method can capture
the distribution of a target image domain which is transformed by the block
encryption algorithm, and upload generator parameters to avoid classifier
sharing with negligible influence on model performance. Furthermore, we apply a
feature extractor to motivate model utility and train it separately from the
classifier. The extensive experimental results and security analyses
demonstrate the superiority of our proposed scheme compared to other
state-of-the-art defense methods. The code is available at
https://github.com/ytingma/PPIDSG.
