  Medical segmentation has grown exponentially through the advent of a fully
convolutional network (FCN), and we have now reached a turning point through
the success of Transformer. However, the different characteristics of the
modality have not been fully integrated into Transformer for medical
segmentation. In this work, we propose the novel hybrid fusion Transformer
(HFTrans) for multisequence MRI image segmentation. We take advantage of the
differences among multimodal MRI sequences and utilize the Transformer layers
to integrate the features extracted from each modality as well as the features
of the early fused modalities. We validate the effectiveness of our
hybrid-fusion method in three-dimensional (3D) medical segmentation.
Experiments on two public datasets, BraTS2020 and MRBrainS18, show that the
proposed method outperforms previous state-of-the-art methods on the task of
brain tumor segmentation and brain structure segmentation.
