  This paper presents a dynamic network rewiring (DNR) method to generate
pruned deep neural network (DNN) models that are robust against adversarial
attacks yet maintain high accuracy on clean images. In particular, the
disclosed DNR method is based on a unified constrained optimization formulation
using a hybrid loss function that merges ultra-high model compression with
robust adversarial training. This training strategy dynamically adjusts
inter-layer connectivity based on per-layer normalized momentum computed from
the hybrid loss function. In contrast to existing robust pruning frameworks
that require multiple training iterations, the proposed learning strategy
achieves an overall target pruning ratio with only a single training iteration
and can be tuned to support both irregular and structured channel pruning. To
evaluate the merits of DNR, experiments were performed with two widely accepted
models, namely VGG16 and ResNet-18, on CIFAR-10, CIFAR-100 as well as with
VGG16 on Tiny-ImageNet. Compared to the baseline uncompressed models, DNR
provides over20x compression on all the datasets with no significant drop in
either clean or adversarial classification accuracy. Moreover, our experiments
show that DNR consistently finds compressed models with better clean and
adversarial image classification performance than what is achievable through
state-of-the-art alternatives.
