  Unsupervised sim-to-real domain adaptation (UDA) for semantic segmentation
aims to improve the real-world test performance of a model trained on simulated
data. It can save the cost of manually labeling data in real-world applications
such as robot vision and autonomous driving. Traditional UDA often assumes that
there are abundant unlabeled real-world data samples available during training
for the adaptation. However, such an assumption does not always hold in
practice owing to the collection difficulty and the scarcity of the data. Thus,
we aim to relieve this need on a large number of real data, and explore the
one-shot unsupervised sim-to-real domain adaptation (OSUDA) and generalization
(OSDG) problem, where only one real-world data sample is available. To remedy
the limited real data knowledge, we first construct the pseudo-target domain by
stylizing the simulated data with the one-shot real data. To mitigate the
sim-to-real domain gap on both the style and spatial structure level and
facilitate the sim-to-real adaptation, we further propose to use class-aware
cross-domain transformers with an intermediate domain randomization strategy to
extract the domain-invariant knowledge, from both the simulated and
pseudo-target data. We demonstrate the effectiveness of our approach for OSUDA
and OSDG on different benchmarks, outperforming the state-of-the-art methods by
a large margin, 10.87, 9.59, 13.05 and 15.91 mIoU on GTA,
SYNTHIA$\rightarrow$Cityscapes, Foggy Cityscapes, respectively.
