  The world faces a shortage of radiologists, leading to longer treatment times
and increased stress, negatively impacting patient safety and workforce morale.
Integrating artificial intelligence to interpret radiographic images and
generate descriptive reports offers a promising solution. However, limited
research exists on generating natural language descriptions for volumetric
medical images. This study introduces a deep learning-based proof of concept
model to accurately identify abnormalities in volumetric CT data and generate
narrative-style reports. Various encoder-decoder models were assessed for their
efficacy in clinically relevant and surrogate tasks. Clinically relevant tasks
involved identifying and describing pulmonary nodules and pleural effusions,
while surrogate tasks involved recognizing and describing artificial
abnormalities such as mirroring, rotation, and lung lobe occlusion. The results
show high accuracy in detecting combinations of artificial abnormalities, with
the best model achieving a classification accuracy of 0.97 on an independent
dataset with a homogeneously distributed 11-class problem. Furthermore, the
best model consistently generated coherent radiology reports in natural
language, with a next-word prediction accuracy of 0.84. Additionally, 65% of
these reports were factually accurate regarding the identified artificial
abnormalities. Unfortunately, these models did not replicate this success for
clinically relevant tasks. Overall, this study provides a working proof of
concept model for a challenge yet to be fully addressed by the scientific
community. Given the success on surrogate tasks, the leap to clinically
relevant tasks seems feasible. Acquiring a significantly larger high-quality
dataset appears to be the most promising path forward, alongside more
computational resources for end-to-end model training.
