  Unsupervised domain adaptation has attracted growing research attention on
semantic segmentation. However, 1) most existing models cannot be directly
applied into lesions transfer of medical images, due to the diverse appearances
of same lesion among different datasets; 2) equal attention has been paid into
all semantic representations instead of neglecting irrelevant knowledge, which
leads to negative transfer of untransferable knowledge. To address these
challenges, we develop a new unsupervised semantic transfer model including two
complementary modules (i.e., T_D and T_F ) for endoscopic lesions segmentation,
which can alternatively determine where and how to explore transferable
domain-invariant knowledge between labeled source lesions dataset (e.g.,
gastroscope) and unlabeled target diseases dataset (e.g., enteroscopy).
Specifically, T_D focuses on where to translate transferable visual information
of medical lesions via residual transferability-aware bottleneck, while
neglecting untransferable visual characterizations. Furthermore, T_F highlights
how to augment transferable semantic features of various lesions and
automatically ignore untransferable representations, which explores
domain-invariant knowledge and in return improves the performance of T_D. To
the end, theoretical analysis and extensive experiments on medical endoscopic
dataset and several non-medical public datasets well demonstrate the
superiority of our proposed model.
