  Federated Learning (FL) has recently emerged as a promising distributed
machine learning framework to preserve clients' privacy, by allowing multiple
clients to upload the gradients calculated from their local data to a central
server. Recent studies find that the exchanged gradients also take the risk of
privacy leakage, e.g., an attacker can invert the shared gradients and recover
sensitive data against an FL system by leveraging pre-trained generative
adversarial networks (GAN) as prior knowledge. However, performing gradient
inversion attacks in the latent space of the GAN model limits their expression
ability and generalizability. To tackle these challenges, we propose
\textbf{G}radient \textbf{I}nversion over \textbf{F}eature \textbf{D}omains
(GIFD), which disassembles the GAN model and searches the feature domains of
the intermediate layers. Instead of optimizing only over the initial latent
code, we progressively change the optimized layer, from the initial latent
space to intermediate layers closer to the output images. In addition, we
design a regularizer to avoid unreal image generation by adding a small ${l_1}$
ball constraint to the searching range. We also extend GIFD to the
out-of-distribution (OOD) setting, which weakens the assumption that the
training sets of GANs and FL tasks obey the same data distribution. Extensive
experiments demonstrate that our method can achieve pixel-level reconstruction
and is superior to the existing methods. Notably, GIFD also shows great
generalizability under different defense strategy settings and batch sizes.
