  Batch Normalization (BatchNorm) is effective for improving the performance
and accelerating the training of deep neural networks. However, it has also
shown to be a cause of adversarial vulnerability, i.e., networks without it are
more robust to adversarial attacks. In this paper, we investigate how BatchNorm
causes this vulnerability and proposed new normalization that is robust to
adversarial attacks. We first observe that adversarial images tend to shift the
distribution of BatchNorm input, and this shift makes train-time estimated
population statistics inaccurate. We hypothesize that these inaccurate
statistics make models with BatchNorm more vulnerable to adversarial attacks.
We prove our hypothesis by replacing train-time estimated statistics with
statistics calculated from the inference-time batch. We found that the
adversarial vulnerability of BatchNorm disappears if we use these statistics.
However, without estimated batch statistics, we can not use BatchNorm in the
practice if large batches of input are not available. To mitigate this, we
propose Robust Normalization (RobustNorm); an adversarially robust version of
BatchNorm. We experimentally show that models trained with RobustNorm perform
better in adversarial settings while retaining all the benefits of BatchNorm.
Code is available at \url{https://github.com/awaisrauf/RobustNorm}.
