  In the field of multimodal segmentation, the correlation between different
modalities can be considered for improving the segmentation results.
Considering the correlation between different MR modalities, in this paper, we
propose a multi-modality segmentation network guided by a novel tri-attention
fusion. Our network includes N model-independent encoding paths with N image
sources, a tri-attention fusion block, a dual-attention fusion block, and a
decoding path. The model independent encoding paths can capture
modality-specific features from the N modalities. Considering that not all the
features extracted from the encoders are useful for segmentation, we propose to
use dual attention based fusion to re-weight the features along the modality
and space paths, which can suppress less informative features and emphasize the
useful ones for each modality at different positions. Since there exists a
strong correlation between different modalities, based on the dual attention
fusion block, we propose a correlation attention module to form the
tri-attention fusion block. In the correlation attention module, a correlation
description block is first used to learn the correlation between modalities and
then a constraint based on the correlation is used to guide the network to
learn the latent correlated features which are more relevant for segmentation.
Finally, the obtained fused feature representation is projected by the decoder
to obtain the segmentation results. Our experiment results tested on BraTS 2018
dataset for brain tumor segmentation demonstrate the effectiveness of our
proposed method.
