  We propose a novel deep convolutional neural network (CNN) based multi-task
learning approach for open-set visual recognition. We combine a classifier
network and a decoder network with a shared feature extractor network within a
multi-task learning framework. We show that this approach results in better
open-set recognition accuracy. In our approach, reconstruction errors from the
decoder network are utilized for open-set rejection. In addition, we model the
tail of the reconstruction error distribution from the known classes using the
statistical Extreme Value Theory to improve the overall performance.
Experiments on multiple image classification datasets are performed and it is
shown that this method can perform significantly better than many competitive
open set recognition algorithms available in the literature. The code will be
made available at: github.com/otkupjnoz/mlosr.
