  To investigate the impact of OOD radiographs on existing chest X-ray
classification models and to increase their robustness against OOD data. The
study employed the commonly used chest X-ray classification model, CheXnet,
trained on the chest X-ray 14 data set, and tested its robustness against OOD
data using three public radiography data sets: IRMA, Bone Age, and MURA, and
the ImageNet data set. To detect OOD data for multi-label classification, we
proposed in-distribution voting (IDV). The OOD detection performance is
measured across data sets using the area under the receiver operating
characteristic curve (AUC) analysis and compared with Mahalanobis-based OOD
detection, MaxLogit, MaxEnergy and self-supervised OOD detection (SS OOD).
Without additional OOD detection, the chest X-ray classifier failed to discard
any OOD images, with an AUC of 0.5. The proposed IDV approach trained on ID
(chest X-ray 14) and OOD data (IRMA and ImageNet) achieved, on average, 0.999
OOD AUC across the three data sets, surpassing all other OOD detection methods.
Mahalanobis-based OOD detection achieved an average OOD detection AUC of 0.982.
IDV trained solely with a few thousand ImageNet images had an AUC 0.913, which
was higher than MaxLogit (0.726), MaxEnergy (0.724), and SS OOD (0.476). The
performance of all tested OOD detection methods did not translate well to
radiography data sets, except Mahalanobis-based OOD detection and the proposed
IDV method. Training solely on ID data led to incorrect classification of OOD
images as ID, resulting in increased false positive rates. IDV substantially
improved the model's ID classification performance, even when trained with data
that will not occur in the intended use case or test set, without additional
inference overhead.
