  Positron emission tomography (PET) combined with computed tomography (CT)
imaging is routinely used in cancer diagnosis and prognosis by providing
complementary information. Automatically segmenting tumors in PET/CT images can
significantly improve examination efficiency. Traditional multi-modal
segmentation solutions mainly rely on concatenation operations for modality
fusion, which fail to effectively model the non-linear dependencies between PET
and CT modalities. Recent studies have investigated various approaches to
optimize the fusion of modality-specific features for enhancing joint
representations. However, modality-specific encoders used in these methods
operate independently, inadequately leveraging the synergistic relationships
inherent in PET and CT modalities, for example, the complementarity between
semantics and structure. To address these issues, we propose a Hierarchical
Adaptive Interaction and Weighting Network termed H2ASeg to explore the
intrinsic cross-modal correlations and transfer potential complementary
information. Specifically, we design a Modality-Cooperative Spatial Attention
(MCSA) module that performs intra- and inter-modal interactions globally and
locally. Additionally, a Target-Aware Modality Weighting (TAMW) module is
developed to highlight tumor-related features within multi-modal features,
thereby refining tumor segmentation. By embedding these modules across
different layers, H2ASeg can hierarchically model cross-modal correlations,
enabling a nuanced understanding of both semantic and structural tumor
features. Extensive experiments demonstrate the superiority of H2ASeg,
outperforming state-of-the-art methods on AutoPet-II and Hecktor2022
benchmarks. The code is released at https://github.com/JinPLu/H2ASeg.
