  We introduce Domain-specific Masks for Generalization, a model for improving
both in-domain and out-of-domain generalization performance. For domain
generalization, the goal is to learn from a set of source domains to produce a
single model that will best generalize to an unseen target domain. As such,
many prior approaches focus on learning representations which persist across
all source domains with the assumption that these domain agnostic
representations will generalize well. However, often individual domains contain
characteristics which are unique and when leveraged can significantly aid
in-domain recognition performance. To produce a model which best generalizes to
both seen and unseen domains, we propose learning domain specific masks. The
masks are encouraged to learn a balance of domain-invariant and domain-specific
features, thus enabling a model which can benefit from the predictive power of
specialized features while retaining the universal applicability of
domain-invariant features. We demonstrate competitive performance compared to
naive baselines and state-of-the-art methods on both PACS and DomainNet.
