  Domain generalization (DG) remains a significant challenge for perception
based on deep neural networks (DNN), where domain shifts occur due to lighting,
weather, or geolocation changes. In this work, we propose VLTSeg to enhance
domain generalization in semantic segmentation, where the network is solely
trained on the source domain and evaluated on unseen target domains. Our method
leverages the inherent semantic robustness of vision-language models. First, by
substituting traditional vision-only backbones with pre-trained encoders from
CLIP and EVA-CLIP as transfer learning setting we find that in the field of DG,
vision-language pre-training significantly outperforms supervised and
self-supervised vision pre-training. We thus propose a new vision-language
approach for domain generalized segmentation, which improves the domain
generalization SOTA by 7.6% mIoU when training on the synthetic GTA5 dataset.
We further show the superior generalization capabilities of vision-language
segmentation models by reaching 76.48% mIoU on the popular Cityscapes-to-ACDC
benchmark, outperforming the previous SOTA approach by 6.9% mIoU on the test
set at the time of writing. Additionally, our approach shows strong in-domain
generalization capabilities indicated by 86.1% mIoU on the Cityscapes test set,
resulting in a shared first place with the previous SOTA on the current
leaderboard at the time of submission.
