  Federated Learning (FL) has emerged as a promising approach for collaborative
model training without sharing private data. However, privacy concerns
regarding information exchanged during FL have received significant research
attention. Gradient Inversion Attacks (GIAs) have been proposed to reconstruct
the private data retained by local clients from the exchanged gradients. While
recovering private data, the data dimensions and the model complexity increase,
which thwart data reconstruction by GIAs. Existing methods adopt prior
knowledge about private data to overcome those challenges. In this paper, we
first observe that GIAs with gradients from a single iteration fail to
reconstruct private data due to insufficient dimensions of leaked gradients,
complex model architectures, and invalid gradient information. We investigate a
Temporal Gradient Inversion Attack with a Robust Optimization framework, called
TGIAs-RO, which recovers private data without any prior knowledge by leveraging
multiple temporal gradients. To eliminate the negative impacts of outliers,
e.g., invalid gradients for collaborative optimization, robust statistics are
proposed. Theoretical guarantees on the recovery performance and robustness of
TGIAs-RO against invalid gradients are also provided. Extensive empirical
results on MNIST, CIFAR10, ImageNet and Reuters 21578 datasets show that the
proposed TGIAs-RO with 10 temporal gradients improves reconstruction
performance compared to state-of-the-art methods, even for large batch sizes
(up to 128), complex models like ResNet18, and large datasets like ImageNet
(224*224 pixels). Furthermore, the proposed attack method inspires further
exploration of privacy-preserving methods in the context of FL.
