  To ensure the out-of-distribution (OOD) generalization performance,
traditional domain generalization (DG) methods resort to training on data from
multiple sources with different underlying distributions. And the success of
those DG methods largely depends on the fact that there are diverse training
distributions. However, it usually needs great efforts to obtain enough
heterogeneous data due to the high expenses, privacy issues or the scarcity of
data. Thus an interesting yet seldom investigated problem arises: how to
improve the OOD generalization performance when the perceived heterogeneity is
limited. In this paper, we instantiate a new framework called few-domain
generalization (FDG), which aims to learn a generalizable model from very few
domains of novel tasks with the knowledge acquired from previous learning
experiences on base tasks. Moreover, we propose a Meta Adaptive Task Sampling
(MATS) procedure to differentiate base tasks according to their semantic and
domain-shift similarity to the novel task. Empirically, we show that the newly
introduced FDG framework can substantially improve the OOD generalization
performance on the novel task and further combining MATS with episodic training
could outperform several state-of-the-art DG baselines on widely used
benchmarks like PACS and DomainNet.
