  Deep Neural Networks (DNN) have improved the quality of several non-safety
related products in the past years. However, before DNNs should be deployed to
safety-critical applications, their robustness needs to be systematically
analyzed. A common challenge for DNNs occurs when input is dissimilar to the
training set, which might lead to high confidence predictions despite proper
knowledge of the input. Several previous studies have proposed to complement
DNNs with a supervisor that detects when inputs are outside the scope of the
network. Most of these supervisors, however, are developed and tested for a
selected scenario using a specific performance metric. In this work, we
emphasize the need to assess and compare the performance of supervisors in a
structured way. We present a framework constituted by four datasets organized
in six test cases combined with seven evaluation metrics. The test cases
provide varying complexity and include data from publicly available sources as
well as a novel dataset consisting of images from simulated driving scenarios.
The latter we plan to make publicly available. Our framework can be used to
support DNN supervisor evaluation, which in turn could be used to motive
development, validation, and deployment of DNNs in safety-critical
applications.
