  This work tackles a central machine learning problem of performance
degradation on out-of-distribution (OOD) test sets. The problem is particularly
salient in medical imaging based diagnosis system that appears to be accurate
but fails when tested in new hospitals/datasets. Recent studies indicate the
system might learn shortcut and non-relevant features instead of generalizable
features, so-called good features. We hypothesize that adversarial training can
eliminate shortcut features whereas saliency guided training can filter out
non-relevant features; both are nuisance features accounting for the
performance degradation on OOD test sets. With that, we formulate a novel model
training scheme for the deep neural network to learn good features for
classification and/or detection tasks ensuring a consistent generalization
performance on OOD test sets. The experimental results qualitatively and
quantitatively demonstrate the superior performance of our method using the
benchmark CXR image data sets on classification tasks.
