  Unsupervised Domain Adaptation (UDA) methods aim to transfer knowledge from a
labeled source domain to an unlabeled target domain. UDA has been extensively
studied in the computer vision literature. Deep networks have been shown to be
vulnerable to adversarial attacks. However, very little focus is devoted to
improving the adversarial robustness of deep UDA models, causing serious
concerns about model reliability. Adversarial Training (AT) has been considered
to be the most successful adversarial defense approach. Nevertheless,
conventional AT requires ground-truth labels to generate adversarial examples
and train models, which limits its effectiveness in the unlabeled target
domain. In this paper, we aim to explore AT to robustify UDA models: How to
enhance the unlabeled data robustness via AT while learning domain-invariant
features for UDA? To answer this question, we provide a systematic study into
multiple AT variants that can potentially be applied to UDA. Moreover, we
propose a novel Adversarially Robust Training method for UDA accordingly,
referred to as ARTUDA. Extensive experiments on multiple adversarial attacks
and UDA benchmarks show that ARTUDA consistently improves the adversarial
robustness of UDA models. Code is available at
https://github.com/shaoyuanlo/ARTUDA
