  Domain Adaptation (DA), the process of effectively adapting task models
learned on one domain, the source, to other related but distinct domains, the
targets, with no or minimal retraining, is typically accomplished using the
process of source-to-target manifold alignment. However, this process often
leads to unsatisfactory adaptation performance, in part because it ignores the
task-specific structure of the data. In this paper, we improve the performance
of DA by introducing a discriminative discrepancy measure which takes advantage
of auxiliary information available in the source and the target domains to
better align the source and target distributions. Specifically, we leverage the
cohesive clustering structure within individual data manifolds, associated with
different tasks, to improve the alignment. This structure is explicit in the
source, where the task labels are available, but is implicit in the target,
making the problem challenging. We address the challenge by devising a deep DA
framework, which combines a new task-driven domain alignment discriminator with
domain regularizers that encourage the shared features as task-specific and
domain invariant, and prompt the task model to be data structure preserving,
guiding its decision boundaries through the low density data regions. We
validate our framework on standard benchmarks, including Digits (MNIST, USPS,
SVHN, MNIST-M), PACS, and VisDA. Our results show that our proposed model
consistently outperforms the state-of-the-art in unsupervised domain
adaptation.
