  We propose a novel unsupervised domain adaptation framework based on
domain-specific batch normalization in deep neural networks. We aim to adapt to
both domains by specializing batch normalization layers in convolutional neural
networks while allowing them to share all other model parameters, which is
realized by a two-stage algorithm. In the first stage, we estimate
pseudo-labels for the examples in the target domain using an external
unsupervised domain adaptation algorithm---for example, MSTN or
CPUA---integrating the proposed domain-specific batch normalization. The second
stage learns the final models using a multi-task classification loss for the
source and target domains. Note that the two domains have separate batch
normalization layers in both stages. Our framework can be easily incorporated
into the domain adaptation techniques based on deep neural networks with batch
normalization layers. We also present that our approach can be extended to the
problem with multiple source domains. The proposed algorithm is evaluated on
multiple benchmark datasets and achieves the state-of-the-art accuracy in the
standard setting and the multi-source domain adaption scenario.
