  Domain generalization (DG) aims to help models trained on a set of source
domains generalize better on unseen target domains. The performances of current
DG methods largely rely on sufficient labeled data, which are usually costly or
unavailable, however. Since unlabeled data are far more accessible, we seek to
explore how unsupervised learning can help deep models generalize across
domains. Specifically, we study a novel generalization problem called
unsupervised domain generalization (UDG), which aims to learn generalizable
models with unlabeled data and analyze the effects of pre-training on DG. In
UDG, models are pretrained with unlabeled data from various source domains
before being trained on labeled source data and eventually tested on unseen
target domains. Then we propose a method named Domain-Aware Representation
LearnING (DARLING) to cope with the significant and misleading heterogeneity
within unlabeled pretraining data and severe distribution shifts between source
and target data. Surprisingly we observe that DARLING can not only
counterbalance the scarcity of labeled data but also further strengthen the
generalization ability of models when the labeled data are insufficient. As a
pretraining approach, DARLING shows superior or comparable performance compared
with ImageNet pretraining protocol even when the available data are unlabeled
and of a vastly smaller amount compared to ImageNet, which may shed light on
improving generalization with large-scale unlabeled data.
