  We conduct investigations on clinical text machine translation by examining
multilingual neural network models using deep learning such as Transformer
based structures. Furthermore, to address the language resource imbalance
issue, we also carry out experiments using a transfer learning methodology
based on massive multilingual pre-trained language models (MMPLMs). The
experimental results on three subtasks including 1) clinical case (CC), 2)
clinical terminology (CT), and 3) ontological concept (OC) show that our models
achieved top-level performances in the ClinSpEn-2022 shared task on
English-Spanish clinical domain data. Furthermore, our expert-based human
evaluations demonstrate that the small-sized pre-trained language model (PLM)
won over the other two extra-large language models by a large margin, in the
clinical domain fine-tuning, which finding was never reported in the field.
Finally, the transfer learning method works well in our experimental setting
using the WMT21fb model to accommodate a new language space Spanish that was
not seen at the pre-training stage within WMT21fb itself, which deserves more
exploitation for clinical knowledge transformation, e.g. to investigate into
more languages. These research findings can shed some light on domain-specific
machine translation development, especially in clinical and healthcare fields.
Further research projects can be carried out based on our work to improve
healthcare text analytics and knowledge transformation. Our data will be openly
available for research purposes at https://github.com/HECTA-UoM/ClinicalNMT
