  Medical image registration is an essential topic in medical image analysis.
In this paper, we propose a method for medical image registration using a
pretrained large language model. We find that using the pretrained large
language model to encode deep features of the medical images in the
registration model can effectively improve image registration accuracy,
indicating the great potential of the large language model in medical image
registration tasks. We use dual encoders to perform deep feature extraction on
image pairs and then input the features into the pretrained large language
model. To adapt the large language model to our registration task, the weights
of the large language model are frozen in the registration model, and an
adapter is utilized to fine-tune the large language model, which aims at (a)
mapping the visual tokens to the language space before the large language model
computing, (b) project the modeled language tokens output from the large
language model to the visual space. Our method combines output features from
the fine-tuned large language model with the features output from each encoder
layer to gradually generate the deformation fields required for registration in
the decoder. To demonstrate the effectiveness of the large prediction model in
registration tasks, we conducted experiments on knee and brain MRI and achieved
state-of-the-art results.
