  When trained on diverse labeled data, machine learning models have proven
themselves to be a powerful tool in all facets of society. However, due to
budget limitations, deliberate or non-deliberate censorship, and other problems
during data collection and curation, the labeled training set might exhibit a
systematic shortage of data for certain groups. We investigate a scenario in
which the absence of certain data is linked to the second level of a two-level
hierarchy in the data. Inspired by the idea of protected groups from
algorithmic fairness, we refer to the partitions carved by this second level as
"subgroups"; we refer to combinations of subgroups and classes, or leaves of
the hierarchy, as "sources". To characterize the problem, we introduce the
concept of classes with incomplete subgroup support. The representational bias
in the training set can give rise to spurious correlations between the classes
and the subgroups which render standard classification models ungeneralizable
to unseen sources. To overcome this bias, we make use of an additional, diverse
but unlabeled dataset, called the "deployment set", to learn a representation
that is invariant to subgroup. This is done by adversarially matching the
support of the training and deployment sets in representation space. In order
to learn the desired invariance, it is paramount that the sets of samples
observed by the discriminator are balanced by class; this is easily achieved
for the training set, but requires using semi-supervised clustering for the
deployment set. We demonstrate the effectiveness of our method with experiments
on several datasets and variants of the problem.
