  Open-set recognition and adversarial defense study two key aspects of deep
learning that are vital for real-world deployment. The objective of open-set
recognition is to identify samples from open-set classes during testing, while
adversarial defense aims to defend the network against images with
imperceptible adversarial perturbations. In this paper, we show that open-set
recognition systems are vulnerable to adversarial attacks. Furthermore, we show
that adversarial defense mechanisms trained on known classes do not generalize
well to open-set samples. Motivated by this observation, we emphasize the need
of an Open-Set Adversarial Defense (OSAD) mechanism. This paper proposes an
Open-Set Defense Network (OSDN) as a solution to the OSAD problem. The proposed
network uses an encoder with feature-denoising layers coupled with a classifier
to learn a noise-free latent feature representation. Two techniques are
employed to obtain an informative latent feature space with the objective of
improving open-set performance. First, a decoder is used to ensure that clean
images can be reconstructed from the obtained latent features. Then,
self-supervision is used to ensure that the latent features are informative
enough to carry out an auxiliary task. We introduce a testing protocol to
evaluate OSAD performance and show the effectiveness of the proposed method in
multiple object classification datasets. The implementation code of the
proposed method is available at: https://github.com/rshaojimmy/ECCV2020-OSAD.
