  Deep networks have produced significant gains for various visual recognition
problems, leading to high impact academic and commercial applications. Recent
work in deep networks highlighted that it is easy to generate images that
humans would never classify as a particular object class, yet networks classify
such images high confidence as that given class - deep network are easily
fooled with images humans do not consider meaningful. The closed set nature of
deep networks forces them to choose from one of the known classes leading to
such artifacts. Recognition in the real world is open set, i.e. the recognition
system should reject unknown/unseen classes at test time. We present a
methodology to adapt deep networks for open set recognition, by introducing a
new model layer, OpenMax, which estimates the probability of an input being
from an unknown class. A key element of estimating the unknown probability is
adapting Meta-Recognition concepts to the activation patterns in the
penultimate layer of the network. OpenMax allows rejection of "fooling" and
unrelated open set images presented to the system; OpenMax greatly reduces the
number of obvious errors made by a deep network. We prove that the OpenMax
concept provides bounded open space risk, thereby formally providing an open
set recognition solution. We evaluate the resulting open set deep networks
using pre-trained networks from the Caffe Model-zoo on ImageNet 2012 validation
data, and thousands of fooling and open set images. The proposed OpenMax model
significantly outperforms open set recognition accuracy of basic deep networks
as well as deep networks with thresholding of SoftMax probabilities.
