  Unsupervised domain adaptive classifcation intends to improve the
classifcation performance on unlabeled target domain. To alleviate the adverse
effect of domain shift, many approaches align the source and target domains in
the feature space. However, a feature is usually taken as a whole for alignment
without explicitly making domain alignment proactively serve the classifcation
task, leading to sub-optimal solution. In this paper, we propose an effective
Task-oriented Alignment (ToAlign) for unsupervised domain adaptation (UDA). We
study what features should be aligned across domains and propose to make the
domain alignment proactively serve classifcation by performing feature
decomposition and alignment under the guidance of the prior knowledge induced
from the classifcation task itself. Particularly, we explicitly decompose a
feature in the source domain into a task-related/discriminative feature that
should be aligned, and a task-irrelevant feature that should be
avoided/ignored, based on the classifcation meta-knowledge. Extensive
experimental results on various benchmarks (e.g., Offce-Home, Visda-2017, and
DomainNet) under different domain adaptation settings demonstrate the
effectiveness of ToAlign which helps achieve the state-of-the-art performance.
The code is publicly available at https://github.com/microsoft/UDA
