  Adversarial training (AT) can help improve the robustness of Vision
Transformers (ViT) against adversarial attacks by intentionally injecting
adversarial examples into the training data. However, this way of adversarial
injection inevitably incurs standard accuracy degradation to some extent,
thereby calling for a trade-off between standard accuracy and robustness.
Besides, the prominent AT solutions are still vulnerable to adaptive attacks.
To tackle such shortcomings, this paper proposes a novel ViT architecture,
including a detector and a classifier bridged by our newly developed adaptive
ensemble. Specifically, we empirically discover that detecting adversarial
examples can benefit from the Guided Backpropagation technique. Driven by this
discovery, a novel Multi-head Self-Attention (MSA) mechanism is introduced to
enhance our detector to sniff adversarial examples. Then, a classifier with two
encoders is employed for extracting visual representations respectively from
clean images and adversarial examples, with our adaptive ensemble to adaptively
adjust the proportion of visual representations from the two encoders for
accurate classification. This design enables our ViT architecture to achieve a
better trade-off between standard accuracy and robustness. Besides, our
adaptive ensemble technique allows us to mask off a random subset of image
patches within input data, boosting our ViT's robustness against adaptive
attacks, while maintaining high standard accuracy. Experimental results exhibit
that our ViT architecture, on CIFAR-10, achieves the best standard accuracy and
adversarial robustness of 90.3% and 49.8%, respectively.
