  With the growing use of transformer-based language models in medicine, it is
unclear how well these models generalize to nuclear medicine which has
domain-specific vocabulary and unique reporting styles. In this study, we
evaluated the value of domain adaptation in nuclear medicine by adapting
language models for the purpose of 5-point Deauville score prediction based on
clinical 18F-fluorodeoxyglucose (FDG) PET/CT reports. We retrospectively
retrieved 4542 text reports and 1664 images for FDG PET/CT lymphoma exams from
2008-2018 in our clinical imaging database. Deauville scores were removed from
the reports and then the remaining text in the reports was used as the model
input. Multiple general-purpose transformer language models were used to
classify the reports into Deauville scores 1-5. We then adapted the models to
the nuclear medicine domain using masked language modeling and assessed its
impact on classification performance. The language models were compared against
vision models, a multimodal vision language model, and a nuclear medicine
physician with seven-fold Monte Carlo cross validation, reported are the mean
and standard deviations. Domain adaption improved all language models. For
example, BERT improved from 61.3% five-class accuracy to 65.7% following domain
adaptation. The best performing model (domain-adapted RoBERTa) achieved a
five-class accuracy of 77.4%, which was better than the physician's performance
(66%), the best vision model's performance (48.1), and was similar to the
multimodal model's performance (77.2). Domain adaptation improved the
performance of large language models in interpreting nuclear medicine text
reports.
