  Synthetic data offers the promise of cheap and bountiful training data for
settings where labeled real-world data is scarce. However, models trained on
synthetic data significantly underperform when evaluated on real-world data. In
this paper, we propose Proportional Amplitude Spectrum Training Augmentation
(PASTA), a simple and effective augmentation strategy to improve out-of-the-box
synthetic-to-real (syn-to-real) generalization performance. PASTA perturbs the
amplitude spectra of synthetic images in the Fourier domain to generate
augmented views. Specifically, with PASTA we propose a structured perturbation
strategy where high-frequency components are perturbed relatively more than the
low-frequency ones. For the tasks of semantic segmentation (GTAV-to-Real),
object detection (Sim10K-to-Real), and object recognition (VisDA-C
Syn-to-Real), across a total of 5 syn-to-real shifts, we find that PASTA
outperforms more complex state-of-the-art generalization methods while being
complementary to the same.
