  Despite recent success, state-of-the-art learning-based models remain highly
vulnerable to input changes such as adversarial examples. In order to obtain
certifiable robustness against such perturbations, recent work considers
Lipschitz-based regularizers or constraints while at the same time increasing
prediction margin. Unfortunately, this comes at the cost of significantly
decreased accuracy. In this paper, we propose a Calibrated Lipschitz-Margin
Loss (CLL) that addresses this issue and improves certified robustness by
tackling two problems: Firstly, commonly used margin losses do not adjust the
penalties to the shrinking output distribution; caused by minimizing the
Lipschitz constant $K$. Secondly, and most importantly, we observe that
minimization of $K$ can lead to overly smooth decision functions. This limits
the model's complexity and thus reduces accuracy. Our CLL addresses these
issues by explicitly calibrating the loss w.r.t. margin and Lipschitz constant,
thereby establishing full control over slack and improving robustness
certificates even with larger Lipschitz constants. On CIFAR-10, CIFAR-100 and
Tiny-ImageNet, our models consistently outperform losses that leave the
constant unattended. On CIFAR-100 and Tiny-ImageNet, CLL improves upon
state-of-the-art deterministic $L_2$ robust accuracies. In contrast to current
trends, we unlock potential of much smaller models without $K=1$ constraints.
