  Regularizing the input gradient has shown to be effective in promoting the
robustness of neural networks. The regularization of the input's Hessian is
therefore a natural next step. A key challenge here is the computational
complexity. Computing the Hessian of inputs is computationally infeasible. In
this paper we propose an efficient algorithm to train deep neural networks with
Hessian operator-norm regularization. We analyze the approach theoretically and
prove that the Hessian operator norm relates to the ability of a neural network
to withstand an adversarial attack. We give a preliminary experimental
evaluation on the MNIST and FMNIST datasets, which demonstrates that the new
regularizer can, indeed, be feasible and, furthermore, that it increases the
robustness of neural networks over input gradient regularization.
