  Automation of medical image interpretation could alleviate bottlenecks in
diagnostic workflows, and has become of particular interest in recent years due
to advancements in natural language processing. Great strides have been made
towards automated radiology report generation via AI, yet ensuring clinical
accuracy in generated reports is a significant challenge, hindering deployment
of such methods in clinical practice. In this work we propose a quality control
framework for assessing the reliability of AI-generated radiology reports with
respect to semantics of diagnostic importance using modular auxiliary auditing
components (AC). Evaluating our pipeline on the MIMIC-CXR dataset, our findings
show that incorporating ACs in the form of disease-classifiers can enable
auditing that identifies more reliable reports, resulting in higher F1 scores
compared to unfiltered generated reports. Additionally, leveraging the
confidence of the AC labels further improves the audit's effectiveness.
