  This paper presents a novel unsupervised domain adaptation framework, called
Synergistic Image and Feature Adaptation (SIFA), to effectively tackle the
problem of domain shift. Domain adaptation has become an important and hot
topic in recent studies on deep learning, aiming to recover performance
degradation when applying the neural networks to new testing domains. Our
proposed SIFA is an elegant learning diagram which presents synergistic fusion
of adaptations from both image and feature perspectives. In particular, we
simultaneously transform the appearance of images across domains and enhance
domain-invariance of the extracted features towards the segmentation task. The
feature encoder layers are shared by both perspectives to grasp their mutual
benefits during the end-to-end learning procedure. Without using any annotation
from the target domain, the learning of our unified model is guided by
adversarial losses, with multiple discriminators employed from various aspects.
We have extensively validated our method with a challenging application of
cross-modality medical image segmentation of cardiac structures. Experimental
results demonstrate that our SIFA model recovers the degraded performance from
17.2% to 73.0%, and outperforms the state-of-the-art methods by a significant
margin.
