  Transformer, the model of choice for natural language processing, has drawn
scant attention from the medical imaging community. Given the ability to
exploit long-term dependencies, transformers are promising to help atypical
convolutional neural networks to overcome their inherent shortcomings of
spatial inductive bias. However, most of recently proposed transformer-based
segmentation approaches simply treated transformers as assisted modules to help
encode global context into convolutional representations. To address this
issue, we introduce nnFormer, a 3D transformer for volumetric medical image
segmentation. nnFormer not only exploits the combination of interleaved
convolution and self-attention operations, but also introduces local and global
volume-based self-attention mechanism to learn volume representations.
Moreover, nnFormer proposes to use skip attention to replace the traditional
concatenation/summation operations in skip connections in U-Net like
architecture. Experiments show that nnFormer significantly outperforms previous
transformer-based counterparts by large margins on three public datasets.
Compared to nnUNet, nnFormer produces significantly lower HD95 and comparable
DSC results. Furthermore, we show that nnFormer and nnUNet are highly
complementary to each other in model ensembling.
