  In recent years Deep Neural Network-based systems are not only increasing in
popularity but also receive growing user trust. However, due to the
closed-world assumption of such systems, they cannot recognize samples from
unknown classes and often induce an incorrect label with high confidence.
Presented work looks at the evaluation of methods for Open Set Recognition,
focusing on the impact of class imbalance, especially in the dichotomy between
known and unknown samples. As an outcome of problem analysis, we present a set
of guidelines for evaluation of methods in this field.
