  Existing unsupervised domain adaptation methods aim to transfer knowledge
from a label-rich source domain to an unlabeled target domain. However,
obtaining labels for some source domains may be very expensive, making complete
labeling as used in prior work impractical. In this work, we investigate a new
domain adaptation scenario with sparsely labeled source data, where only a few
examples in the source domain have been labeled, while the target domain is
unlabeled. We show that when labeled source examples are limited, existing
methods often fail to learn discriminative features applicable for both source
and target domains. We propose a novel Cross-Domain Self-supervised (CDS)
learning approach for domain adaptation, which learns features that are not
only domain-invariant but also class-discriminative. Our self-supervised
learning method captures apparent visual similarity with in-domain
self-supervision in a domain adaptive manner and performs cross-domain feature
matching with across-domain self-supervision. In extensive experiments with
three standard benchmark datasets, our method significantly boosts performance
of target accuracy in the new target domain with few source labels and is even
helpful on classical domain adaptation scenarios.
