  Flatness of the loss surface not only correlates positively with
generalization but is also related to adversarial robustness, since
perturbations of inputs relate non-linearly to perturbations of weights. In
this paper, we empirically analyze the relation between adversarial examples
and relative flatness with respect to the parameters of one layer. We observe a
peculiar property of adversarial examples: during an iterative first-order
white-box attack, the flatness of the loss surface measured around the
adversarial example first becomes sharper until the label is flipped, but if we
keep the attack running it runs into a flat uncanny valley where the label
remains flipped. We find this phenomenon across various model architectures and
datasets. Our results also extend to large language models (LLMs), but due to
the discrete nature of the input space and comparatively weak attacks, the
adversarial examples rarely reach a truly flat region. Most importantly, this
phenomenon shows that flatness alone cannot explain adversarial robustness
unless we can also guarantee the behavior of the function around the examples.
We theoretically connect relative flatness to adversarial robustness by
bounding the third derivative of the loss surface, underlining the need for
flatness in combination with a low global Lipschitz constant for a robust
model.
