  We use information-theoretic tools to derive a novel analysis of Multi-source
Domain Adaptation (MDA) from the representation learning perspective.
Concretely, we study joint distribution alignment for supervised MDA with few
target labels and unsupervised MDA with pseudo labels, where the latter is
relatively hard and less commonly studied. We further provide
algorithm-dependent generalization bounds for these two settings, where the
generalization is characterized by the mutual information between the
parameters and the data. Then we propose a novel deep MDA algorithm, implicitly
addressing the target shift through joint alignment. Finally, the mutual
information bounds are extended to this algorithm providing a non-vacuous
gradient-norm estimation. The proposed algorithm has comparable performance to
the state-of-the-art on target-shifted MDA benchmark with improved memory
efficiency.
