  In this paper, we propose a new key-based defense focusing on both efficiency
and robustness. Although the previous key-based defense seems effective in
defending against adversarial examples, carefully designed adaptive attacks can
bypass the previous defense, and it is difficult to train the previous defense
on large datasets like ImageNet. We build upon the previous defense with two
major improvements: (1) efficient training and (2) optional randomization. The
proposed defense utilizes one or more secret patch embeddings and classifier
heads with a pre-trained isotropic network. When more than one secret
embeddings are used, the proposed defense enables randomization on inference.
Experiments were carried out on the ImageNet dataset, and the proposed defense
was evaluated against an arsenal of state-of-the-art attacks, including
adaptive ones. The results show that the proposed defense achieves a high
robust accuracy and a comparable clean accuracy compared to the previous
key-based defense.
