  During the forward pass of Deep Neural Networks (DNNs), inputs gradually
transformed from low-level features to high-level conceptual labels. While
features at different layers could summarize the important factors of the
inputs at varying levels, modern out-of-distribution (OOD) detection methods
mostly focus on utilizing their ending layer features. In this paper, we
proposed a novel layer-adaptive OOD detection framework (LA-OOD) for DNNs that
can fully utilize the intermediate layers' outputs. Specifically, instead of
training a unified OOD detector at a fixed ending layer, we train multiple
One-Class SVM OOD detectors simultaneously at the intermediate layers to
exploit the full spectrum characteristics encoded at varying depths of DNNs. We
develop a simple yet effective layer-adaptive policy to identify the best layer
for detecting each potential OOD example. LA-OOD can be applied to any existing
DNNs and does not require access to OOD samples during the training. Using
three DNNs of varying depth and architectures, our experiments demonstrate that
LA-OOD is robust against OODs of varying complexity and can outperform
state-of-the-art competitors by a large margin on some real-world datasets.
