  Federated learning (FL) aims to protect data privacy by enabling clients to
build machine learning models collaboratively without sharing their private
data. Recent works demonstrate that information exchanged during FL is subject
to gradient-based privacy attacks, and consequently, a variety of
privacy-preserving methods have been adopted to thwart such attacks. However,
these defensive methods either introduce orders of magnitude more computational
and communication overheads (e.g., with homomorphic encryption) or incur
substantial model performance losses in terms of prediction accuracy (e.g.,
with differential privacy). In this work, we propose $\textsc{FedCG}$, a novel
federated learning method that leverages conditional generative adversarial
networks to achieve high-level privacy protection while still maintaining
competitive model performance. $\textsc{FedCG}$ decomposes each client's local
network into a private extractor and a public classifier and keeps the
extractor local to protect privacy. Instead of exposing extractors,
$\textsc{FedCG}$ shares clients' generators with the server for aggregating
clients' shared knowledge, aiming to enhance the performance of each client's
local networks. Extensive experiments demonstrate that $\textsc{FedCG}$ can
achieve competitive model performance compared with FL baselines, and privacy
analysis shows that $\textsc{FedCG}$ has a high-level privacy-preserving
capability. Code is available at https://github.com/yankang18/FedCG
