  Recently, domain adaptation has become a hot research area with lots of
applications. The goal is to adapt a model trained in one domain to another
domain with scarce annotated data. We propose a simple yet effective method
based on self-supervised learning that outperforms or is on par with most
state-of-the-art algorithms, e.g. adversarial domain adaptation. Our method
involves two phases: predicting random rotations (self-supervised) on the
target domain along with correct labels for the source domain (supervised), and
then using self-distillation on the target domain. Our simple method achieves
state-of-the-art results on semi-supervised domain adaptation on DomainNet
dataset.
  Further, we observe that the unlabeled target datasets of popular domain
adaptation benchmarks do not contain any categories apart from testing
categories. We believe this introduces a bias that does not exist in many real
applications. We show that removing this bias from the unlabeled data results
in a large drop in performance of state-of-the-art methods, while our simple
method is relatively robust.
