  Domain adversarial training has shown its effective capability for finding
domain invariant feature representations and been successfully adopted for
various domain adaptation tasks. However, recent advances of large models
(e.g., vision transformers) and emerging of complex adaptation scenarios (e.g.,
DomainNet) make adversarial training being easily biased towards source domain
and hardly adapted to target domain. The reason is twofold: relying on large
amount of labelled data from source domain for large model training and lacking
of labelled data from target domain for fine-tuning. Existing approaches widely
focused on either enhancing discriminator or improving the training stability
for the backbone networks. Due to unbalanced competition between the feature
extractor and the discriminator during the adversarial training, existing
solutions fail to function well on complex datasets. To address this issue, we
proposed a novel contrastive adversarial training (CAT) approach that leverages
the labeled source domain samples to reinforce and regulate the feature
generation for target domain. Typically, the regulation forces the target
feature distribution being similar to the source feature distribution. CAT
addressed three major challenges in adversarial learning: 1) ensure the feature
distributions from two domains as indistinguishable as possible for the
discriminator, resulting in a more robust domain-invariant feature generation;
2) encourage target samples moving closer to the source in the feature space,
reducing the requirement for generalizing classifier trained on the labeled
source domain to unlabeled target domain; 3) avoid directly aligning unpaired
source and target samples within mini-batch. CAT can be easily plugged into
existing models and exhibits significant performance improvements.
