  Out of distribution (OOD) detection is a crucial part of making machine
learning systems robust. The ImageNet-O dataset is an important tool in testing
the robustness of ImageNet trained deep neural networks that are widely used
across a variety of systems and applications. We aim to perform a comparative
analysis of OOD detection methods on ImageNet-O, a first of its kind dataset
with a label distribution different than that of ImageNet, that has been
created to aid research in OOD detection for ImageNet models. As this dataset
is fairly new, we aim to provide a comprehensive benchmarking of some of the
current state of the art OOD detection methods on this novel dataset. This
benchmarking covers a variety of model architectures, settings where we haves
prior access to the OOD data versus when we don't, predictive score based
approaches, deep generative approaches to OOD detection, and more.
