  While Unsupervised Domain Adaptation (UDA) algorithms, i.e., there are only
labeled data from source domains, have been actively studied in recent years,
most algorithms and theoretical results focus on Single-source Unsupervised
Domain Adaptation (SUDA). However, in the practical scenario, labeled data can
be typically collected from multiple diverse sources, and they might be
different not only from the target domain but also from each other. Thus,
domain adapters from multiple sources should not be modeled in the same way.
Recent deep learning based Multi-source Unsupervised Domain Adaptation (MUDA)
algorithms focus on extracting common domain-invariant representations for all
domains by aligning distribution of all pairs of source and target domains in a
common feature space. However, it is often very hard to extract the same
domain-invariant representations for all domains in MUDA. In addition, these
methods match distributions without considering domain-specific decision
boundaries between classes. To solve these problems, we propose a new framework
with two alignment stages for MUDA which not only respectively aligns the
distributions of each pair of source and target domains in multiple specific
feature spaces, but also aligns the outputs of classifiers by utilizing the
domain-specific decision boundaries. Extensive experiments demonstrate that our
method can achieve remarkable results on popular benchmark datasets for image
classification.
