  Unsupervised domain adaptation (UDA) seeks to alleviate the problem of domain
shift between the distribution of unlabeled data from the target domain w.r.t.
labeled data from the source domain. While the single-target UDA scenario is
well studied in the literature, Multi-Target Domain Adaptation (MTDA) remains
largely unexplored despite its practical importance, e.g., in multi-camera
video-surveillance applications. The MTDA problem can be addressed by adapting
one specialized model per target domain, although this solution is too costly
in many real-world applications. Blending multiple targets for MTDA has been
proposed, yet this solution may lead to a reduction in model specificity and
accuracy. In this paper, we propose a novel unsupervised MTDA approach to train
a CNN that can generalize well across multiple target domains. Our
Multi-Teacher MTDA (MT-MTDA) method relies on multi-teacher knowledge
distillation (KD) to iteratively distill target domain knowledge from multiple
teachers to a common student. The KD process is performed in a progressive
manner, where the student is trained by each teacher on how to perform UDA for
a specific target, instead of directly learning domain adapted features.
Finally, instead of combining the knowledge from each teacher, MT-MTDA
alternates between teachers that distill knowledge, thereby preserving the
specificity of each target (teacher) when learning to adapt to the student.
MT-MTDA is compared against state-of-the-art methods on several challenging UDA
benchmarks, and empirical results show that our proposed model can provide a
considerably higher level of accuracy across multiple target domains. Our code
is available at: https://github.com/LIVIAETS/MT-MTDA
