  Inspired by the recent success of Transformers for Natural Language
Processing and vision Transformer for Computer Vision, many researchers in the
medical imaging community have flocked to Transformer-based networks for
various main stream medical tasks such as classification, segmentation, and
estimation. In this study, we analyze, two recently published Transformer-based
network architectures for the task of multimodal head-and-tumor segmentation
and compare their performance to the de facto standard 3D segmentation network
- the nnU-Net. Our results showed that modeling long-range dependencies may be
helpful in cases where large structures are present and/or large field of view
is needed. However, for small structures such as head-and-neck tumor, the
convolution-based U-Net architecture seemed to perform well, especially when
training dataset is small and computational resource is limited.
