  To defend deep neural networks from adversarial attacks, adversarial training
has been drawing increasing attention for its effectiveness. However, the
accuracy and robustness resulting from the adversarial training are limited by
the architecture, because adversarial training improves accuracy and robustness
by adjusting the weight connection affiliated to the architecture. In this
work, we propose ARNAS to search for accurate and robust architectures for
adversarial training. First we design an accurate and robust search space, in
which the placement of the cells and the proportional relationship of the
filter numbers are carefully determined. With the design, the architectures can
obtain both accuracy and robustness by deploying accurate and robust structures
to their sensitive positions, respectively. Then we propose a differentiable
multi-objective search strategy, performing gradient descent towards directions
that are beneficial for both natural loss and adversarial loss, thus the
accuracy and robustness can be guaranteed at the same time. We conduct
comprehensive experiments in terms of white-box attacks, black-box attacks, and
transferability. Experimental results show that the searched architecture has
the strongest robustness with the competitive accuracy, and breaks the
traditional idea that NAS-based architectures cannot transfer well to complex
tasks in robustness scenarios. By analyzing outstanding architectures searched,
we also conclude that accurate and robust neural architectures tend to deploy
different structures near the input and output, which has great practical
significance on both hand-crafting and automatically designing of accurate and
robust architectures.
