  Sobolev loss is used when training a network to approximate the values and
derivatives of a target function at a prescribed set of input points. Recent
works have demonstrated its successful applications in various tasks such as
distillation or synthetic gradient prediction. In this work we prove that an
overparameterized two-layer relu neural network trained on the Sobolev loss
with gradient flow from random initialization can fit any given function values
and any given directional derivatives, under a separation condition on the
input data.
