  In this paper, we prove that in the overparametrized regime, deep neural
network provide universal approximations and can interpolate any data set, as
long as the activation function is locally in $L^1(\RR)$ and not an affine
function.
  Additionally, if the activation function is smooth and such an interpolation
networks exists, then the set of parameters which interpolate forms a manifold.
Furthermore, we give a characterization of the Hessian of the loss function
evaluated at the interpolation points.
  In the last section, we provide a practical probabilistic method of finding
such a point under general conditions on the activation function.
