  In this paper, we study the formalism of unsupervised multi-class domain
adaptation (multi-class UDA), which underlies a few recent algorithms whose
learning objectives are only motivated empirically. Multi-Class Scoring
Disagreement (MCSD) divergence is presented by aggregating the absolute margin
violations in multi-class classification, and this proposed MCSD is able to
fully characterize the relations between any pair of multi-class scoring
hypotheses. By using MCSD as a measure of domain distance, we develop a new
domain adaptation bound for multi-class UDA; its data-dependent, probably
approximately correct bound is also developed that naturally suggests
adversarial learning objectives to align conditional feature distributions
across source and target domains. Consequently, an algorithmic framework of
Multi-class Domain-adversarial learning Networks (McDalNets) is developed, and
its different instantiations via surrogate learning objectives either coincide
with or resemble a few recently popular methods, thus (partially) underscoring
their practical effectiveness. Based on our identical theory for multi-class
UDA, we also introduce a new algorithm of Domain-Symmetric Networks (SymmNets),
which is featured by a novel adversarial strategy of domain confusion and
discrimination. SymmNets affords simple extensions that work equally well under
the problem settings of either closed set, partial, or open set UDA. We conduct
careful empirical studies to compare different algorithms of McDalNets and our
newly introduced SymmNets. Experiments verify our theoretical analysis and show
the efficacy of our proposed SymmNets. In addition, we have made our
implementation code publicly available.
