  The cost of large scale data collection and annotation often makes the
application of machine learning algorithms to new tasks or datasets
prohibitively expensive. One approach circumventing this cost is training
models on synthetic data where annotations are provided automatically. Despite
their appeal, such models often fail to generalize from synthetic to real
images, necessitating domain adaptation algorithms to manipulate these models
before they can be successfully applied. Existing approaches focus either on
mapping representations from one domain to the other, or on learning to extract
features that are invariant to the domain from which they were extracted.
However, by focusing only on creating a mapping or shared representation
between the two domains, they ignore the individual characteristics of each
domain. We suggest that explicitly modeling what is unique to each domain can
improve a model's ability to extract domain-invariant features. Inspired by
work on private-shared component analysis, we explicitly learn to extract image
representations that are partitioned into two subspaces: one component which is
private to each domain and one which is shared across domains. Our model is
trained not only to perform the task we care about in the source domain, but
also to use the partitioned representation to reconstruct the images from both
domains. Our novel architecture results in a model that outperforms the
state-of-the-art on a range of unsupervised domain adaptation scenarios and
additionally produces visualizations of the private and shared representations
enabling interpretation of the domain adaptation process.
