  Adversarial attacks that generate small L_p-norm perturbations to mislead
classifiers have limited success in black-box settings and with unseen
classifiers. These attacks are also not robust to defenses that use denoising
filters and to adversarial training procedures. Instead, adversarial attacks
that generate unrestricted perturbations are more robust to defenses, are
generally more successful in black-box settings and are more transferable to
unseen classifiers. However, unrestricted perturbations may be noticeable to
humans. In this paper, we propose a content-based black-box adversarial attack
that generates unrestricted perturbations by exploiting image semantics to
selectively modify colors within chosen ranges that are perceived as natural by
humans. We show that the proposed approach, ColorFool, outperforms in terms of
success rate, robustness to defense frameworks and transferability, five
state-of-the-art adversarial attacks on two different tasks, scene and object
classification, when attacking three state-of-the-art deep neural networks
using three standard datasets. The source code is available at
https://github.com/smartcameras/ColorFool.
