  In this paper, we study the properties of robust nonparametric estimation
using deep neural networks for regression models with heavy tailed error
distributions. We establish the non-asymptotic error bounds for a class of
robust nonparametric regression estimators using deep neural networks with ReLU
activation under suitable smoothness conditions on the regression function and
mild conditions on the error term. In particular, we only assume that the error
distribution has a finite p-th moment with p greater than one. We also show
that the deep robust regression estimators are able to circumvent the curse of
dimensionality when the distribution of the predictor is supported on an
approximate lower-dimensional set. An important feature of our error bound is
that, for ReLU neural networks with network width and network size (number of
parameters) no more than the order of the square of the dimensionality d of the
predictor, our excess risk bounds depend sub-linearly on d. Our assumption
relaxes the exact manifold support assumption, which could be restrictive and
unrealistic in practice. We also relax several crucial assumptions on the data
distribution, the target regression function and the neural networks required
in the recent literature. Our simulation studies demonstrate that the robust
methods can significantly outperform the least squares method when the errors
have heavy-tailed distributions and illustrate that the choice of loss function
is important in the context of deep nonparametric regression.
