  This work concerns the development of deep networks that are certifiably
robust to adversarial attacks. Joint robust classification-detection was
recently introduced as a certified defense mechanism, where adversarial
examples are either correctly classified or assigned to the "abstain" class. In
this work, we show that such a provable framework can benefit by extension to
networks with multiple explicit abstain classes, where the adversarial examples
are adaptively assigned to those. We show that naively adding multiple abstain
classes can lead to "model degeneracy", then we propose a regularization
approach and a training method to counter this degeneracy by promoting full use
of the multiple abstain classes. Our experiments demonstrate that the proposed
approach consistently achieves favorable standard vs. robust verified accuracy
tradeoffs, outperforming state-of-the-art algorithms for various choices of
number of abstain classes.
