  In this paper, we propose a model-based characterization of neural networks
to detect novel input types and conditions. Novelty detection is crucial to
identify abnormal inputs that can significantly degrade the performance of
machine learning algorithms. Majority of existing studies have focused on
activation-based representations to detect abnormal inputs, which limits the
characterization of abnormality from a data perspective. However, a model
perspective can also be informative in terms of the novelties and
abnormalities. To articulate the significance of the model perspective in
novelty detection, we utilize backpropagated gradients. We conduct a
comprehensive analysis to compare the representation capability of gradients
with that of activation and show that the gradients outperform the activation
in novel class and condition detection. We validate our approach using four
image recognition datasets including MNIST, Fashion-MNIST, CIFAR-10, and
CURE-TSR. We achieve a significant improvement on all four datasets with an
average AUROC of 0.953, 0.918, 0.582, and 0.746, respectively.
