  Image classifiers based on deep neural networks suffer from harassment caused
by adversarial examples. Two defects exist in black-box iterative attacks that
generate adversarial examples by incrementally adjusting the noise-adding
direction for each step. On the one hand, existing iterative attacks add noises
monotonically along the direction of gradient ascent, resulting in a lack of
diversity and adaptability of the generated iterative trajectories. On the
other hand, it is trivial to perform adversarial attack by adding excessive
noises, but currently there is no refinement mechanism to squeeze redundant
noises. In this work, we propose Curls & Whey black-box attack to fix the above
two defects. During Curls iteration, by combining gradient ascent and descent,
we `curl' up iterative trajectories to integrate more diversity and
transferability into adversarial examples. Curls iteration also alleviates the
diminishing marginal effect in existing iterative attacks. The Whey
optimization further squeezes the `whey' of noises by exploiting the robustness
of adversarial perturbation. Extensive experiments on Imagenet and
Tiny-Imagenet demonstrate that our approach achieves impressive decrease on
noise magnitude in l2 norm. Curls & Whey attack also shows promising
transferability against ensemble models as well as adversarially trained
models. In addition, we extend our attack to the targeted misclassification,
effectively reducing the difficulty of targeted attacks under black-box
condition.
