  The performance of deep neural networks is often attributed to their
automated, task-related feature construction. It remains an open question,
though, why this leads to solutions with good generalization, even in cases
where the number of parameters is larger than the number of samples. Back in
the 90s, Hochreiter and Schmidhuber observed that flatness of the loss surface
around a local minimum correlates with low generalization error. For several
flatness measures, this correlation has been empirically validated. However, it
has recently been shown that existing measures of flatness cannot theoretically
be related to generalization due to a lack of invariance with respect to
reparameterizations. We propose a natural modification of existing flatness
measures that results in invariance to reparameterization.
