  Adversarial examples have pointed out Deep Neural Networks vulnerability to
small local noise. It has been shown that constraining their Lipschitz constant
should enhance robustness, but make them harder to learn with classical loss
functions. We propose a new framework for binary classification, based on
optimal transport, which integrates this Lipschitz constraint as a theoretical
requirement. We propose to learn 1-Lipschitz networks using a new loss that is
an hinge regularized version of the Kantorovich-Rubinstein dual formulation for
the Wasserstein distance estimation. This loss function has a direct
interpretation in terms of adversarial robustness together with certifiable
robustness bound. We also prove that this hinge regularized version is still
the dual formulation of an optimal transportation problem, and has a solution.
We also establish several geometrical properties of this optimal solution, and
extend the approach to multi-class problems. Experiments show that the proposed
approach provides the expected guarantees in terms of robustness without any
significant accuracy drop. The adversarial examples, on the proposed models,
visibly and meaningfully change the input providing an explanation for the
classification.
