  Methods which utilize the outputs or feature representations of predictive
models have emerged as promising approaches for out-of-distribution (OOD)
detection of image inputs. However, these methods struggle to detect OOD inputs
that share nuisance values (e.g. background) with in-distribution inputs. The
detection of shared-nuisance out-of-distribution (SN-OOD) inputs is
particularly relevant in real-world applications, as anomalies and
in-distribution inputs tend to be captured in the same settings during
deployment. In this work, we provide a possible explanation for SN-OOD
detection failures and propose nuisance-aware OOD detection to address them.
Nuisance-aware OOD detection substitutes a classifier trained via empirical
risk minimization and cross-entropy loss with one that 1. is trained under a
distribution where the nuisance-label relationship is broken and 2. yields
representations that are independent of the nuisance under this distribution,
both marginally and conditioned on the label. We can train a classifier to
achieve these objectives using Nuisance-Randomized Distillation (NuRD), an
algorithm developed for OOD generalization under spurious correlations. Output-
and feature-based nuisance-aware OOD detection perform substantially better
than their original counterparts, succeeding even when detection based on
domain generalization algorithms fails to improve performance.
