  Neural networks are prone to misclassify slightly modified input images.
Recently, many defences have been proposed, but none have improved the
robustness of neural networks consistently. Here, we propose to use adversarial
attacks as a function evaluation to search for neural architectures that can
resist such attacks automatically. Experiments on neural architecture search
algorithms from the literature show that although accurate, they are not able
to find robust architectures. A significant reason for this lies in their
limited search space. By creating a novel neural architecture search with
options for dense layers to connect with convolution layers and vice-versa as
well as the addition of concatenation layers in the search, we were able to
evolve an architecture that is inherently accurate on adversarial samples.
Interestingly, this inherent robustness of the evolved architecture rivals
state-of-the-art defences such as adversarial training while being trained only
on the non-adversarial samples. Moreover, the evolved architecture makes use of
some peculiar traits which might be useful for developing even more robust
ones. Thus, the results here confirm that more robust architectures exist as
well as opens up a new realm of feasibilities for the development and
exploration of neural networks.
  Code available at http://bit.ly/RobustArchitectureSearch.
