  The dynamics of DNNs during gradient descent is described by the so-called
Neural Tangent Kernel (NTK). In this article, we show that the NTK allows one
to gain precise insight into the Hessian of the cost of DNNs. When the NTK is
fixed during training, we obtain a full characterization of the asymptotics of
the spectrum of the Hessian, at initialization and during training. In the
so-called mean-field limit, where the NTK is not fixed during training, we
describe the first two moments of the Hessian at initialization.
