  One popular group of defense techniques against adversarial attacks is based
on injecting stochastic noise into the network. The main source of robustness
of such stochastic defenses however is often due to the obfuscation of the
gradients, offering a false sense of security. Since most of the popular
adversarial attacks are optimization-based, obfuscated gradients reduce their
attacking ability, while the model is still susceptible to stronger or
specifically tailored adversarial attacks. Recently, five characteristics have
been identified, which are commonly observed when the improvement in robustness
is mainly caused by gradient obfuscation. It has since become a trend to use
these five characteristics as a sufficient test, to determine whether or not
gradient obfuscation is the main source of robustness. However, these
characteristics do not perfectly characterize all existing cases of gradient
obfuscation, and therefore can not serve as a basis for a conclusive test. In
this work, we present a counterexample, showing this test is not sufficient for
concluding that gradient obfuscation is not the main cause of improvements in
robustness.
