  In safety-critical but computationally resource-constrained applications,
deep learning faces two key challenges: lack of robustness against adversarial
attacks and large neural network size (often millions of parameters). While the
research community has extensively explored the use of robust training and
network pruning independently to address one of these challenges, only a few
recent works have studied them jointly. However, these works inherit a
heuristic pruning strategy that was developed for benign training, which
performs poorly when integrated with robust training techniques, including
adversarial training and verifiable robust training. To overcome this
challenge, we propose to make pruning techniques aware of the robust training
objective and let the training objective guide the search for which connections
to prune. We realize this insight by formulating the pruning objective as an
empirical risk minimization problem which is solved efficiently using SGD. We
demonstrate that our approach, titled HYDRA, achieves compressed networks with
state-of-the-art benign and robust accuracy, simultaneously. We demonstrate the
success of our approach across CIFAR-10, SVHN, and ImageNet dataset with four
robust training techniques: iterative adversarial training, randomized
smoothing, MixTrain, and CROWN-IBP. We also demonstrate the existence of highly
robust sub-networks within non-robust networks. Our code and compressed
networks are publicly available at
\url{https://github.com/inspire-group/compactness-robustness}.
