  Due to domain shifts, machine learning systems typically struggle to
generalize well to new domains that differ from those of training data, which
is what domain generalization (DG) aims to address. Although a variety of DG
methods have been proposed, most of them fall short in interpretability and
require domain labels, which are not available in many real-world scenarios.
This paper presents a novel DG method, called HMOE: Hypernetwork-based Mixture
of Experts (MoE), which does not rely on domain labels and is more
interpretable. MoE proves effective in identifying heterogeneous patterns in
data. For the DG problem, heterogeneity arises exactly from domain shifts. HMOE
employs hypernetworks taking vectors as input to generate the weights of
experts, which promotes knowledge sharing among experts and enables the
exploration of their similarities in a low-dimensional vector space. We
benchmark HMOE against other DG methods under a fair evaluation framework --
DomainBed. Our extensive experiments show that HMOE can effectively separate
mixed-domain data into distinct clusters that are surprisingly more consistent
with human intuition than original domain labels. Using self-learned domain
information, HMOE achieves state-of-the-art results on most datasets and
significantly surpasses other DG methods in average accuracy across all
datasets.
