  Synthetic data is emerging as a promising solution to the scalability issue
of supervised deep learning, especially when real data are difficult to acquire
or hard to annotate. Synthetic data generation, however, can itself be
prohibitively expensive when domain experts have to manually and painstakingly
oversee the process. Moreover, neural networks trained on synthetic data often
do not perform well on real data because of the domain gap. To solve these
challenges, we propose Sim2SG, a self-supervised automatic scene generation
technique for matching the distribution of real data. Importantly, Sim2SG does
not require supervision from the real-world dataset, thus making it applicable
in situations for which such annotations are difficult to obtain. Sim2SG is
designed to bridge both the content and appearance gaps, by matching the
content of real data, and by matching the features in the source and target
domains. We select scene graph (SG) generation as the downstream task, due to
the limited availability of labeled datasets. Experiments demonstrate
significant improvements over leading baselines in reducing the domain gap both
qualitatively and quantitatively, on several synthetic datasets as well as the
real-world KITTI dataset.
