  Deep Learning has been shown to be particularly vulnerable to adversarial
samples. To combat adversarial strategies, numerous defensive techniques have
been proposed. Among these, a promising approach is to use randomness in order
to make the classification process unpredictable and presumably harder for the
adversary to control. In this paper, we study the effectiveness of randomized
defenses against adversarial samples. To this end, we categorize existing
state-of-the-art adversarial strategies into three attacker models of
increasing strength, namely blackbox, graybox, and whitebox (a.k.a.~adaptive)
attackers. We also devise a lightweight randomization strategy for image
classification based on feature squeezing, that consists of pre-processing the
classifier input by embedding randomness within each feature, before applying
feature squeezing. We evaluate the proposed defense and compare it to other
randomized techniques in the literature via thorough experiments. Our results
indeed show that careful integration of randomness can be effective against
both graybox and blackbox attacks without significantly degrading the accuracy
of the underlying classifier. However, our experimental results offer strong
evidence that in the present form such randomization techniques cannot deter a
whitebox adversary that has access to all classifier parameters and has full
knowledge of the defense. Our work thoroughly and empirically analyzes the
impact of randomization techniques against all classes of adversarial
strategies.
