  Due to the lack of properly annotated medical data, exploring the
generalization capability of the deep model is becoming a public concern.
Zero-shot learning (ZSL) has emerged in recent years to equip the deep model
with the ability to recognize unseen classes. However, existing studies mainly
focus on natural images, which utilize linguistic models to extract auxiliary
information for ZSL. It is impractical to apply the natural image ZSL solutions
directly to medical images, since the medical terminology is very
domain-specific, and it is not easy to acquire linguistic models for the
medical terminology. In this work, we propose a new paradigm of ZSL
specifically for medical images utilizing cross-modality information. We make
three main contributions with the proposed paradigm. First, we extract the
prior knowledge about the segmentation targets, called relation prototypes,
from the prior model and then propose a cross-modality adaptation module to
inherit the prototypes to the zero-shot model. Second, we propose a relation
prototype awareness module to make the zero-shot model aware of information
contained in the prototypes. Last but not least, we develop an inheritance
attention module to recalibrate the relation prototypes to enhance the
inheritance process. The proposed framework is evaluated on two public
cross-modality datasets including a cardiac dataset and an abdominal dataset.
Extensive experiments show that the proposed framework significantly
outperforms the state of the arts.
