  The extraction of structured clinical information from free-text radiology
reports in the form of radiology graphs has been demonstrated to be a valuable
approach for evaluating the clinical correctness of report-generation methods.
However, the direct generation of radiology graphs from chest X-ray (CXR)
images has not been attempted. To address this gap, we propose a novel approach
called Prior-RadGraphFormer that utilizes a transformer model with prior
knowledge in the form of a probabilistic knowledge graph (PKG) to generate
radiology graphs directly from CXR images. The PKG models the statistical
relationship between radiology entities, including anatomical structures and
medical observations. This additional contextual information enhances the
accuracy of entity and relation extraction. The generated radiology graphs can
be applied to various downstream tasks, such as free-text or structured reports
generation and multi-label classification of pathologies. Our approach
represents a promising method for generating radiology graphs directly from CXR
images, and has significant potential for improving medical image analysis and
clinical decision-making.
