  In this study, we aimed to determine if fine-tuned large language models
(LLMs) can generate accurate, personalized impressions for whole-body PET
reports. Twelve language models were trained on a corpus of PET reports using
the teacher-forcing algorithm, with the report findings as input and the
clinical impressions as reference. An extra input token encodes the reading
physician's identity, allowing models to learn physician-specific reporting
styles. Our corpus comprised 37,370 retrospective PET reports collected from
our institution between 2010 and 2022. To identify the best LLM, 30 evaluation
metrics were benchmarked against quality scores from two nuclear medicine (NM)
physicians, with the most aligned metrics selecting the model for expert
evaluation. In a subset of data, model-generated impressions and original
clinical impressions were assessed by three NM physicians according to 6
quality dimensions (3-point scale) and an overall utility score (5-point
scale). Each physician reviewed 12 of their own reports and 12 reports from
other physicians. Bootstrap resampling was used for statistical analysis. Of
all evaluation metrics, domain-adapted BARTScore and PEGASUSScore showed the
highest Spearman's rank correlations (0.568 and 0.563) with physician
preferences. Based on these metrics, the fine-tuned PEGASUS model was selected
as the top LLM. When physicians reviewed PEGASUS-generated impressions in their
own style, 89% were considered clinically acceptable, with a mean utility score
of 4.08 out of 5. Physicians rated these personalized impressions as comparable
in overall utility to the impressions dictated by other physicians (4.03,
P=0.41). In conclusion, personalized impressions generated by PEGASUS were
clinically useful, highlighting its potential to expedite PET reporting.
