  What sorts of structure might enable a learner to discover classes from
unlabeled data? Traditional approaches rely on feature-space similarity and
heroic assumptions on the data. In this paper, we introduce unsupervised
learning under Latent Label Shift (LLS), where we have access to unlabeled data
from multiple domains such that the label marginals $p_d(y)$ can shift across
domains but the class conditionals $p(\mathbf{x}|y)$ do not. This work
instantiates a new principle for identifying classes: elements that shift
together group together. For finite input spaces, we establish an isomorphism
between LLS and topic modeling: inputs correspond to words, domains to
documents, and labels to topics. Addressing continuous data, we prove that when
each label's support contains a separable region, analogous to an anchor word,
oracle access to $p(d|\mathbf{x})$ suffices to identify $p_d(y)$ and
$p_d(y|\mathbf{x})$ up to permutation. Thus motivated, we introduce a practical
algorithm that leverages domain-discriminative models as follows: (i) push
examples through domain discriminator $p(d|\mathbf{x})$; (ii) discretize the
data by clustering examples in $p(d|\mathbf{x})$ space; (iii) perform
non-negative matrix factorization on the discrete data; (iv) combine the
recovered $p(y|d)$ with the discriminator outputs $p(d|\mathbf{x})$ to compute
$p_d(y|x) \; \forall d$. With semi-synthetic experiments, we show that our
algorithm can leverage domain information to improve upon competitive
unsupervised classification methods. We reveal a failure mode of standard
unsupervised classification methods when feature-space similarity does not
indicate true groupings, and show empirically that our method better handles
this case. Our results establish a deep connection between distribution shift
and topic modeling, opening promising lines for future work.
