  Despite achieving remarkable success in various domains, recent studies have
uncovered the vulnerability of deep neural networks to adversarial
perturbations, creating concerns on model generalizability and new threats such
as prediction-evasive misclassification or stealthy reprogramming. Among
different defense proposals, stochastic network defenses such as random neuron
activation pruning or random perturbation to layer inputs are shown to be
promising for attack mitigation. However, one critical drawback of current
defenses is that the robustness enhancement is at the cost of noticeable
performance degradation on legitimate data, e.g., large drop in test accuracy.
This paper is motivated by pursuing for a better trade-off between adversarial
robustness and test accuracy for stochastic network defenses. We propose
Defense Efficiency Score (DES), a comprehensive metric that measures the gain
in unsuccessful attack attempts at the cost of drop in test accuracy of any
defense. To achieve a better DES, we propose hierarchical random switching
(HRS), which protects neural networks through a novel randomization scheme. A
HRS-protected model contains several blocks of randomly switching channels to
prevent adversaries from exploiting fixed model structures and parameters for
their malicious purposes. Extensive experiments show that HRS is superior in
defending against state-of-the-art white-box and adaptive adversarial
misclassification attacks. We also demonstrate the effectiveness of HRS in
defending adversarial reprogramming, which is the first defense against
adversarial programs. Moreover, in most settings the average DES of HRS is at
least 5X higher than current stochastic network defenses, validating its
significantly improved robustness-accuracy trade-off.
