  Deep neural networks have achieved remarkable performance in various
applications but are extremely vulnerable to adversarial perturbation. The most
representative and promising methods that can enhance model robustness, such as
adversarial training and its variants, substantially degrade model accuracy on
benign samples, limiting practical utility. Although incorporating extra
training data can alleviate the trade-off to a certain extent, it remains
unsolved to achieve both robustness and accuracy under limited training data.
Here, we demonstrate the feasibility of overcoming the trade-off, by developing
an adversarial feature stacking (AFS) model, which combines multiple
independent feature extractors with varied levels of robustness and accuracy.
Theoretical analysis is further conducted, and general principles for the
selection of basic feature extractors are provided. We evaluate the AFS model
on CIFAR-10 and CIFAR-100 datasets with strong adaptive attack methods,
significantly advancing the state-of-the-art in terms of the trade-off. The AFS
model achieves a benign accuracy improvement of ~6% on CIFAR-10 and ~10% on
CIFAR-100 with comparable or even stronger robustness than the state-of-the-art
adversarial training methods.
