  Computer vision systems for automatic image categorization have become
accurate and reliable enough that they can run continuously for days or even
years as components of real-world commercial applications. A major open problem
in this context, however, is quality control. Good classification performance
can only be expected if systems run under the specific conditions, in
particular data distributions, that they were trained for. Surprisingly, none
of the currently used deep network architectures has a built-in functionality
that could detect if a network operates on data from a distribution that it was
not trained for and potentially trigger a warning to the human users. In this
work, we describe KS(conf), a procedure for detecting such outside of the
specifications operation. Building on statistical insights, its main step is
the applications of a classical Kolmogorov-Smirnov test to the distribution of
predicted confidence values. We show by extensive experiments using ImageNet,
AwA2 and DAVIS data on a variety of ConvNets architectures that KS(conf)
reliably detects out-of-specs situations. It furthermore has a number of
properties that make it an excellent candidate for practical deployment: it is
easy to implement, adds almost no overhead to the system, works with all
networks, including pretrained ones, and requires no a priori knowledge about
how the data distribution could change.
