  Thyroid nodule segmentation is a crucial step in the diagnostic procedure of
physicians and computer-aided diagnosis systems. Mostly, current studies treat
segmentation and diagnosis as independent tasks without considering the
correlation between these tasks. The sequence steps of these independent tasks
in computer-aided diagnosis systems may lead to the accumulation of errors.
Therefore, it is worth combining them as a whole through exploring the
relationship between thyroid nodule segmentation and diagnosis. According to
the thyroid imaging reporting and data system (TI-RADS), the assessment of
shape and margin characteristics is the prerequisite for the discrimination of
benign and malignant thyroid nodules. These characteristics can be observed in
the thyroid nodule segmentation masks. Inspired by the diagnostic procedure of
TI-RADS, this paper proposes a shape-margin knowledge augmented network
(SkaNet) for simultaneously thyroid nodule segmentation and diagnosis. Due to
the similarity in visual features between segmentation and diagnosis, SkaNet
shares visual features in the feature extraction stage and then utilizes a
dual-branch architecture to perform thyroid nodule segmentation and diagnosis
tasks simultaneously. To enhance effective discriminative features, an
exponential mixture module is devised, which incorporates convolutional feature
maps and self-attention maps by exponential weighting. Then, SkaNet is jointly
optimized by a knowledge augmented multi-task loss function with a constraint
penalty term. It embeds shape and margin characteristics through numerical
computation and models the relationship between the thyroid nodule diagnosis
results and segmentation masks.
