  Automatic classification of active tuberculosis from chest X-ray images has
the potential to save lives, especially in low- and mid-income countries where
skilled human experts can be scarce. Given the lack of available labeled data
to train such systems and the unbalanced nature of publicly available datasets,
we argue that the reliability of deep learning models is limited, even if they
can be shown to obtain perfect classification accuracy on the test data. One
way of evaluating the reliability of such systems is to ensure that models use
the same regions of input images for predictions as medical experts would. In
this paper, we show that pre-training a deep neural network on a large-scale
proxy task, as well as using mixed objective optimization network (MOON), a
technique to balance different classes during pre-training and fine-tuning, can
improve the alignment of decision foundations between models and experts, as
compared to a model directly trained on the target dataset. At the same time,
these approaches keep perfect classification accuracy according to the area
under the receiver operating characteristic curve (AUROC) on the test set, and
improve generalization on an independent, unseen dataset. For the purpose of
reproducibility, our source code is made available online.
