  We propose a novel transformer model, capable of segmenting medical images of
varying modalities. Challenges posed by the fine grained nature of medical
image analysis mean that the adaptation of the transformer for their analysis
is still at nascent stages. The overwhelming success of the UNet lay in its
ability to appreciate the fine-grained nature of the segmentation task, an
ability which existing transformer based models do not currently posses. To
address this shortcoming, we propose The Fully Convolutional Transformer (FCT),
which builds on the proven ability of Convolutional Neural Networks to learn
effective image representations, and combines them with the ability of
Transformers to effectively capture long-term dependencies in its inputs. The
FCT is the first fully convolutional Transformer model in medical imaging
literature. It processes its input in two stages, where first, it learns to
extract long range semantic dependencies from the input image, and then learns
to capture hierarchical global attributes from the features. FCT is compact,
accurate and robust. Our results show that it outperforms all existing
transformer architectures by large margins across multiple medical image
segmentation datasets of varying data modalities without the need for any
pre-training. FCT outperforms its immediate competitor on the ACDC dataset by
1.3%, on the Synapse dataset by 4.4%, on the Spleen dataset by 1.2% and on ISIC
2017 dataset by 1.1% on the dice metric, with up to five times fewer
parameters. Our code, environments and models will be available via GitHub.
