  Successful unsupervised domain adaptation is guaranteed only under strong
assumptions such as covariate shift and overlap between input domains. The
latter is often violated in high-dimensional applications like image
classification which, despite this limitation, continues to serve as
inspiration and benchmark for algorithm development. In this work, we show that
training-time access to side information in the form of auxiliary variables can
help relax restrictions on input variables and increase the sample efficiency
of learning at the cost of collecting a richer variable set. As this
information is assumed available only during training, not in deployment, we
call this problem unsupervised domain adaptation by learning using privileged
information (DALUPI). To solve this problem, we propose a simple two-stage
learning algorithm, inspired by our analysis of the expected error in the
target domain, and a practical end-to-end variant for image classification. We
propose three evaluation tasks based on classification of entities in photos
and anomalies in medical images with different types of available privileged
information (binary attributes and single or multiple regions of interest). We
demonstrate across these tasks that using privileged information in learning
can reduce errors in domain transfer compared to baselines, be robust to
spurious correlations in the source domain, and increase sample efficiency.
