  In many cases, neural network classifiers are likely to be exposed to input
data that is outside of their training distribution data. Samples from outside
the distribution may be classified as an existing class with high probability
by softmax-based classifiers; such incorrect classifications affect the
performance of the classifiers and the applications/systems that depend on
them. Previous research aimed at distinguishing training distribution data from
out-of-distribution data (OOD) has proposed detectors that are external to the
classification method. We present Gaussian isolation machine (GIM), a novel
hybrid (generative-discriminative) classifier aimed at solving the problem
arising when OOD data is encountered. The GIM is based on a neural network and
utilizes a new loss function that imposes a distribution on each of the trained
classes in the neural network's output space, which can be approximated by a
Gaussian. The proposed GIM's novelty lies in its discriminative performance and
generative capabilities, a combination of characteristics not usually seen in a
single classifier. The GIM achieves state-of-the-art classification results on
image recognition and sentiment analysis benchmarking datasets and can also
deal with OOD inputs.
