  Open set recognition requires a classifier to detect samples not belonging to
any of the classes in its training set. Existing methods fit a probability
distribution to the training samples on their embedding space and detect
outliers according to this distribution. The embedding space is often obtained
from a discriminative classifier. However, such discriminative representation
focuses only on known classes, which may not be critical for distinguishing the
unknown classes. We argue that the representation space should be jointly
learned from the inlier classifier and the density estimator (served as an
outlier detector). We propose the OpenHybrid framework, which is composed of an
encoder to encode the input data into a joint embedding space, a classifier to
classify samples to inlier classes, and a flow-based density estimator to
detect whether a sample belongs to the unknown category. A typical problem of
existing flow-based models is that they may assign a higher likelihood to
outliers. However, we empirically observe that such an issue does not occur in
our experiments when learning a joint representation for discriminative and
generative components. Experiments on standard open set benchmarks also reveal
that an end-to-end trained OpenHybrid model significantly outperforms
state-of-the-art methods and flow-based baselines.
