  We study a new highly-practical problem setting that enables
resource-constrained edge devices to adapt a pre-trained model to their local
data distributions. Recognizing that device's data are likely to come from
multiple latent domains that include a mixture of unlabelled domain-relevant
and domain-irrelevant examples, we focus on the comparatively under-studied
problem of latent domain adaptation. Considering limitations of edge devices,
we aim to only use a pre-trained model and adapt it in a feed-forward way,
without using back-propagation and without access to the source data. Modelling
these realistic constraints bring us to the novel and practically important
problem setting of feed-forward latent domain adaptation. Our solution is to
meta-learn a network capable of embedding the mixed-relevance target dataset
and dynamically adapting inference for target examples using cross-attention.
The resulting framework leads to consistent improvements over strong ERM
baselines. We also show that our framework sometimes even improves on the upper
bound of domain-supervised adaptation, where only domain-relevant instances are
provided for adaptation. This suggests that human annotated domain labels may
not always be optimal, and raises the possibility of doing better through
automated instance selection.
