  Deep learning models are vulnerable to adversarial examples, which can fool a
target classifier by imposing imperceptible perturbations onto natural
examples. In this work, we consider the practical and challenging
decision-based black-box adversarial setting, where the attacker can only
acquire the final classification labels by querying the target model without
access to the model's details. Under this setting, existing works often rely on
heuristics and exhibit unsatisfactory performance. To better understand the
rationality of these heuristics and the limitations of existing methods, we
propose to automatically discover decision-based adversarial attack algorithms.
In our approach, we construct a search space using basic mathematical
operations as building blocks and develop a random search algorithm to
efficiently explore this space by incorporating several pruning techniques and
intuitive priors inspired by program synthesis works. Although we use a small
and fast model to efficiently evaluate attack algorithms during the search,
extensive experiments demonstrate that the discovered algorithms are simple yet
query-efficient when transferred to larger normal and defensive models on the
CIFAR-10 and ImageNet datasets. They achieve comparable or better performance
than the state-of-the-art decision-based attack methods consistently.
