  Unsupervised domain adaptation aims to learn a model of classifier for
unlabeled samples on the target domain, given training data of labeled samples
on the source domain. Impressive progress is made recently by learning
invariant features via domain-adversarial training of deep networks. In spite
of the recent progress, domain adaptation is still limited in achieving the
invariance of feature distributions at a finer category level. To this end, we
propose in this paper a new domain adaptation method called Domain-Symmetric
Networks (SymNets). The proposed SymNet is based on a symmetric design of
source and target task classifiers, based on which we also construct an
additional classifier that shares with them its layer neurons. To train the
SymNet, we propose a novel adversarial learning objective whose key design is
based on a two-level domain confusion scheme, where the category-level
confusion loss improves over the domain-level one by driving the learning of
intermediate network features to be invariant at the corresponding categories
of the two domains. Both domain discrimination and domain confusion are
implemented based on the constructed additional classifier. Since target
samples are unlabeled, we also propose a scheme of cross-domain training to
help learn the target classifier. Careful ablation studies show the efficacy of
our proposed method. In particular, based on commonly used base networks, our
SymNets achieve the new state of the art on three benchmark domain adaptation
datasets.
