  In this paper we develop a new perspective on generalization of neural
networks by proposing and investigating the concept of a neural network
stiffness. We measure how stiff a network is by looking at how a small gradient
step in the network's parameters on one example affects the loss on another
example. Higher stiffness suggests that a network is learning features that
generalize. In particular, we study how stiffness depends on 1) class
membership, 2) distance between data points in the input space, 3) training
iteration, and 4) learning rate. We present experiments on MNIST, FASHION
MNIST, and CIFAR-10/100 using fully-connected and convolutional neural
networks, as well as on a transformer-based NLP model. We demonstrate the
connection between stiffness and generalization, and observe its dependence on
learning rate. When training on CIFAR-100, the stiffness matrix exhibits a
coarse-grained behavior indicative of the model's awareness of super-class
membership. In addition, we measure how stiffness between two data points
depends on their mutual input-space distance, and establish the concept of a
dynamical critical length -- a distance below which a parameter update based on
a data point influences its neighbors.
