  Multimodal positron emission tomography-computed tomography (PET-CT) is used
routinely in the assessment of cancer. PET-CT combines the high sensitivity for
tumor detection with PET and anatomical information from CT. Tumor segmentation
is a critical element of PET-CT but at present, there is not an accurate
automated segmentation method. Segmentation tends to be done manually by
different imaging experts and it is labor-intensive and prone to errors and
inconsistency. Previous automated segmentation methods largely focused on
fusing information that is extracted separately from the PET and CT modalities,
with the underlying assumption that each modality contains complementary
information. However, these methods do not fully exploit the high PET tumor
sensitivity that can guide the segmentation. We introduce a multimodal spatial
attention module (MSAM) that automatically learns to emphasize regions (spatial
areas) related to tumors and suppress normal regions with physiologic
high-uptake. The resulting spatial attention maps are subsequently employed to
target a convolutional neural network (CNN) for segmentation of areas with
higher tumor likelihood. Our MSAM can be applied to common backbone
architectures and trained end-to-end. Our experimental results on two clinical
PET-CT datasets of non-small cell lung cancer (NSCLC) and soft tissue sarcoma
(STS) validate the effectiveness of the MSAM in these different cancer types.
We show that our MSAM, with a conventional U-Net backbone, surpasses the
state-of-the-art lung tumor segmentation approach by a margin of 7.6% in Dice
similarity coefficient (DSC).
