  Generalization theory has been established for sparse deep neural networks
under high-dimensional regime. Beyond generalization, parameter estimation is
also important since it is crucial for variable selection and interpretability
of deep neural networks. Current theoretical studies concerning parameter
estimation mainly focus on two-layer neural networks, which is due to the fact
that the convergence of parameter estimation heavily relies on the regularity
of the Hessian matrix, while the Hessian matrix of deep neural networks is
highly singular. To avoid the unidentifiability of deep neural networks in
parameter estimation, we propose to conduct nonparametric estimation of partial
derivatives with respect to inputs. We first show that model convergence of
sparse deep neural networks is guaranteed in that the sample complexity only
grows with the logarithm of the number of parameters or the input dimension
when the $\ell_{1}$-norm of parameters is well constrained. Then by bounding
the norm and the divergence of partial derivatives, we establish that the
convergence rate of nonparametric estimation of partial derivatives scales as
$\mathcal{O}(n^{-1/4})$, a rate which is slower than the model convergence rate
$\mathcal{O}(n^{-1/2})$. To the best of our knowledge, this study combines
nonparametric estimation and parametric sparse deep neural networks for the
first time. As nonparametric estimation of partial derivatives is of great
significance for nonlinear variable selection, the current results show the
promising future for the interpretability of deep neural networks.
