  Robustness to adversarial attacks was shown to require a larger model
capacity, and thus a larger memory footprint. In this paper, we introduce an
approach to obtain robust yet compact models by pruning randomly-initialized
binary networks. Unlike adversarial training, which learns the model
parameters, we initialize the model parameters as either +1 or -1, keep them
fixed, and find a subnetwork structure that is robust to attacks. Our method
confirms the Strong Lottery Ticket Hypothesis in the presence of adversarial
attacks, and extends this to binary networks. Furthermore, it yields more
compact networks with competitive performance than existing works by 1)
adaptively pruning different network layers; 2) exploiting an effective binary
initialization scheme; 3) incorporating a last batch normalization layer to
improve training stability. Our experiments demonstrate that our approach not
only always outperforms the state-of-the-art robust binary networks, but also
can achieve accuracy better than full-precision ones on some datasets. Finally,
we show the structured patterns of our pruned binary networks.
