  In this paper, we develop a general theory for adaptive nonparametric
estimation of the mean function of a non-stationary and nonlinear time series
model using deep neural networks (DNNs). We first consider two types of DNN
estimators, non-penalized and sparse-penalized DNN estimators, and establish
their generalization error bounds for general non-stationary time series. We
then derive minimax lower bounds for estimating mean functions belonging to a
wide class of nonlinear autoregressive (AR) models that include nonlinear
generalized additive AR, single index, and threshold AR models. Building upon
the results, we show that the sparse-penalized DNN estimator is adaptive and
attains the minimax optimal rates up to a poly-logarithmic factor for many
nonlinear AR models. Through numerical simulations, we demonstrate the
usefulness of the DNN methods for estimating nonlinear AR models with intrinsic
low-dimensional structures and discontinuous or rough mean functions, which is
consistent with our theory.
