  Deep neural networks have been a prevailing technique in the field of medical
image processing. However, the most popular convolutional neural networks
(CNNs) based methods for medical image segmentation are imperfect because they
model long-range dependencies by stacking layers or enlarging filters.
Transformers and the self-attention mechanism are recently proposed to
effectively learn long-range dependencies by modeling all pairs of word-to-word
attention regardless of their positions. The idea has also been extended to the
computer vision field by creating and treating image patches as embeddings.
Considering the computation complexity for whole image self-attention, current
transformer-based models settle for a rigid partitioning scheme that
potentially loses informative relations. Besides, current medical transformers
model global context on full resolution images, leading to unnecessary
computation costs. To address these issues, we developed a novel method to
integrate multi-scale attention and CNN feature extraction using a pyramidal
network architecture, namely Pyramid Medical Transformer (PMTrans). The PMTrans
captured multi-range relations by working on multi-resolution images. An
adaptive partitioning scheme was implemented to retain informative relations
and to access different receptive fields efficiently. Experimental results on
three medical image datasets (gland segmentation, MoNuSeg, and HECKTOR
datasets) showed that PMTrans outperformed the latest CNN-based and
transformer-based models for medical image segmentation.
