  Though unsupervised domain adaptation (UDA) has achieved very impressive
progress recently, it remains a great challenge due to missing target
annotations and the rich discrepancy between source and target distributions.
We propose Spectral UDA (SUDA), an effective and efficient UDA technique that
works in the spectral space and can generalize across different visual
recognition tasks. SUDA addresses the UDA challenges from two perspectives.
First, it introduces a spectrum transformer (ST) that mitigates inter-domain
discrepancies by enhancing domain-invariant spectra while suppressing
domain-variant spectra of source and target samples simultaneously. Second, it
introduces multi-view spectral learning that learns useful unsupervised
representations by maximizing mutual information among multiple ST-generated
spectral views of each target sample. Extensive experiments show that SUDA
achieves superior accuracy consistently across different visual tasks in object
detection, semantic segmentation and image classification. Additionally, SUDA
also works with the transformer-based network and achieves state-of-the-art
performance on object detection.
