  Training a deep learning model with artificially generated data can be an
alternative when training data are scarce, yet it suffers from poor
generalization performance due to a large domain gap. In this paper, we
characterize the domain gap by using a causal framework for data generation. We
assume that the real and synthetic data have common content variables but
different style variables. Thus, a model trained on synthetic dataset might
have poor generalization as the model learns the nuisance style variables. To
that end, we propose causal invariance learning which encourages the model to
learn a style-invariant representation that enhances the syn-to-real
generalization. Furthermore, we propose a simple yet effective feature
distillation method that prevents catastrophic forgetting of semantic knowledge
of the real domain. In sum, we refer to our method as Guided Causal Invariant
Syn-to-real Generalization that effectively improves the performance of
syn-to-real generalization. We empirically verify the validity of proposed
methods, and especially, our method achieves state-of-the-art on visual
syn-to-real domain generalization tasks such as image classification and
semantic segmentation.
