  Medical image registration is a fundamental and critical task in medical
image analysis. With the rapid development of deep learning, convolutional
neural networks (CNN) have dominated the medical image registration field. Due
to the disadvantage of the local receptive field of CNN, some recent
registration methods have focused on using transformers for non-local
registration. However, the standard Transformer has a vast number of parameters
and high computational complexity, which causes Transformer can only be applied
at the bottom of the registration models. As a result, only coarse information
is available at the lowest resolution, limiting the contribution of Transformer
in their models. To address these challenges, we propose a convolution-based
efficient multi-head self-attention (CEMSA) block, which reduces the parameters
of the traditional Transformer and captures local spatial context information
for reducing semantic ambiguity in the attention mechanism. Based on the
proposed CEMSA, we present a novel Symmetric Transformer-based model
(SymTrans). SymTrans employs the Transformer blocks in the encoder and the
decoder respectively to model the long-range spatial cross-image relevance. We
apply SymTrans to the displacement field and diffeomorphic registration.
Experimental results show that our proposed method achieves state-of-the-art
performance in image registration. Our code is publicly available at
\url{https://github.com/MingR-Ma/SymTrans}.
