  Recent advances in generative vision-language models (VLMs) have exciting
potential implications for AI in radiology, yet VLMs are also known to produce
hallucinations, nonsensical text, and other unwanted behaviors that can waste
clinicians' time and cause patient harm. Drawing on recent work on direct
preference optimization (DPO), we propose a simple method for modifying the
behavior of pretrained VLMs performing radiology report generation by
suppressing unwanted types of generations. We apply our method to the
prevention of hallucinations of prior exams, addressing a long-established
problem behavior in models performing chest X-ray report generation. Across our
experiments, we find that DPO fine-tuning achieves a 3.2-4.8x reduction in
lines hallucinating prior exams while maintaining model performance on clinical
accuracy metrics. Our work is, to the best of our knowledge, the first work to
apply DPO to medical VLMs, providing a data- and compute- efficient way to
suppress problem behaviors while maintaining overall clinical accuracy.
