  Massively multilingual pre-trained language models (MMPLMs) are developed in
recent years demonstrating superpowers and the pre-knowledge they acquire for
downstream tasks. This work investigates whether MMPLMs can be applied to
clinical domain machine translation (MT) towards entirely unseen languages via
transfer learning. We carry out an experimental investigation using Meta-AI's
MMPLMs ``wmt21-dense-24-wide-en-X and X-en (WMT21fb)'' which were pre-trained
on 7 language pairs and 14 translation directions including English to Czech,
German, Hausa, Icelandic, Japanese, Russian, and Chinese, and the opposite
direction. We fine-tune these MMPLMs towards English-\textit{Spanish} language
pair which \textit{did not exist at all} in their original pre-trained corpora
both implicitly and explicitly. We prepare carefully aligned \textit{clinical}
domain data for this fine-tuning, which is different from their original mixed
domain knowledge. Our experimental result shows that the fine-tuning is very
successful using just 250k well-aligned in-domain EN-ES segments for three
sub-task translation testings: clinical cases, clinical terms, and ontology
concepts. It achieves very close evaluation scores to another MMPLM NLLB from
Meta-AI, which included Spanish as a high-resource setting in the pre-training.
To the best of our knowledge, this is the first work on using MMPLMs towards
\textit{clinical domain transfer-learning NMT} successfully for totally unseen
languages during pre-training.
