  [18F]-Fluorodeoxyglucose (FDG) positron emission tomography - computed
tomography (PET-CT) has become the imaging modality of choice for diagnosing
many cancers. Co-learning complementary PET-CT imaging features is a
fundamental requirement for automatic tumor segmentation and for developing
computer aided cancer diagnosis systems. In this study, we propose a
hyper-connected transformer (HCT) network that integrates a transformer network
(TN) with a hyper connected fusion for multi-modality PET-CT images. The TN was
leveraged for its ability to provide global dependencies in image feature
learning, which was achieved by using image patch embeddings with a
self-attention mechanism to capture image-wide contextual information. We
extended the single-modality definition of TN with multiple TN based branches
to separately extract image features. We also introduced a hyper connected
fusion to fuse the contextual and complementary image features across multiple
transformers in an iterative manner. Our results with two clinical datasets
show that HCT achieved better performance in segmentation accuracy when
compared to the existing methods.
