  Open set recognition (OSR) assumes unknown instances appear out of the blue
at the inference time. The main challenge of OSR is that the response of models
for unknowns is totally unpredictable. Furthermore, the diversity of open set
makes it harder since instances have different difficulty levels. Therefore, we
present a novel framework, DIfficulty-Aware Simulator (DIAS), that generates
fakes with diverse difficulty levels to simulate the real world. We first
investigate fakes from generative adversarial network (GAN) in the classifier's
viewpoint and observe that these are not severely challenging. This leads us to
define the criteria for difficulty by regarding samples generated with GANs
having moderate-difficulty. To produce hard-difficulty examples, we introduce
Copycat, imitating the behavior of the classifier. Furthermore, moderate- and
easy-difficulty samples are also yielded by our modified GAN and Copycat,
respectively. As a result, DIAS outperforms state-of-the-art methods with both
metrics of AUROC and F-score. Our code is available at
https://github.com/wjun0830/Difficulty-Aware-Simulator.
