  We introduce a new approach to prediction in graphical models with
latent-shift adaptation, i.e., where source and target environments differ in
the distribution of an unobserved confounding latent variable. Previous work
has shown that as long as "concept" and "proxy" variables with appropriate
dependence are observed in the source environment, the latent-associated
distributional changes can be identified, and target predictions adapted
accurately. However, practical estimation methods do not scale well when the
observations are complex and high-dimensional, even if the confounding latent
is categorical. Here we build upon a recently proposed probabilistic
unsupervised learning framework, the recognition-parametrised model (RPM), to
recover low-dimensional, discrete latents from image observations. Applied to
the problem of latent shifts, our novel form of RPM identifies causal latent
structure in the source environment, and adapts properly to predict in the
target. We demonstrate results in settings where predictor and proxy are
high-dimensional images, a context to which previous methods fail to scale.
