  Open set recognition (OSR) is the problem of classifying the known classes,
meanwhile identifying the unknown classes when the collected samples cannot
exhaust all the classes. There are many applications for the OSR problem. For
instance, the frequently emerged new malware classes require a system that can
classify the known classes and identify the unknown malware classes. In this
paper, we propose an add-on extension for loss functions in neural networks to
address the OSR problem. Our loss extension leverages the neural network to
find polar representations for the known classes so that the representations of
the known and the unknown classes become more effectively separable. Our
contributions include: First, we introduce an extension that can be
incorporated into different loss functions to find more discriminative
representations. Second, we show that the proposed extension can significantly
improve the performances of two different types of loss functions on datasets
from two different domains. Third, we show that with the proposed extension,
one loss function outperforms the others in terms of training time and model
accuracy.
