  Vision transformers are effective deep learning models for vision tasks,
including medical image segmentation. However, they lack efficiency and
translational invariance, unlike convolutional neural networks (CNNs). To model
long-range interactions in 3D brain lesion segmentation, we propose an
all-convolutional transformer block variant of the U-Net architecture. We
demonstrate that our model provides the greatest compromise in three factors:
performance competitive with the state-of-the-art; parameter efficiency of a
CNN; and the favourable inductive biases of a transformer. Our public
implementation is available at https://github.com/liamchalcroft/MDUNet .
