  With the of advent rich classification models and high computational power
visual recognition systems have found many operational applications.
Recognition in the real world poses multiple challenges that are not apparent
in controlled lab environments. The datasets are dynamic and novel categories
must be continuously detected and then added. At prediction time, a trained
system has to deal with myriad unseen categories. Operational systems require
minimum down time, even to learn. To handle these operational issues, we
present the problem of Open World recognition and formally define it. We prove
that thresholding sums of monotonically decreasing functions of distances in
linearly transformed feature space can balance "open space risk" and empirical
risk. Our theory extends existing algorithms for open world recognition. We
present a protocol for evaluation of open world recognition systems. We present
the Nearest Non-Outlier (NNO) algorithm which evolves model efficiently, adding
object categories incrementally while detecting outliers and managing open
space risk. We perform experiments on the ImageNet dataset with 1.2M+ images to
validate the effectiveness of our method on large scale visual recognition
tasks. NNO consistently yields superior results on open world recognition.
