  We propose associative domain adaptation, a novel technique for end-to-end
domain adaptation with neural networks, the task of inferring class labels for
an unlabeled target domain based on the statistical properties of a labeled
source domain. Our training scheme follows the paradigm that in order to
effectively derive class labels for the target domain, a network should produce
statistically domain invariant embeddings, while minimizing the classification
error on the labeled source domain. We accomplish this by reinforcing
associations between source and target data directly in embedding space. Our
method can easily be added to any existing classification network with no
structural and almost no computational overhead. We demonstrate the
effectiveness of our approach on various benchmarks and achieve
state-of-the-art results across the board with a generic convolutional neural
network architecture not specifically tuned to the respective tasks. Finally,
we show that the proposed association loss produces embeddings that are more
effective for domain adaptation compared to methods employing maximum mean
discrepancy as a similarity measure in embedding space.
