  A crucial requirement for reliable deployment of deep learning models for
safety-critical applications is the ability to identify out-of-distribution
(OOD) data points, samples which differ from the training data and on which a
model might underperform. Previous work has attempted to tackle this problem
using uncertainty estimation techniques. However, there is empirical evidence
that a large family of these techniques do not detect OOD reliably in
classification tasks.
  This paper gives a theoretical explanation for said experimental findings and
illustrates it on synthetic data. We prove that such techniques are not able to
reliably identify OOD samples in a classification setting, since their level of
confidence is generalized to unseen areas of the feature space. This result
stems from the interplay between the representation of ReLU networks as
piece-wise affine transformations, the saturating nature of activation
functions like softmax, and the most widely-used uncertainty metrics.
