  In this paper, we consider estimation and inference for the unknown
parameters and function involved in a class of generalized hierarchical models.
Such models are of great interest in the literature of neural networks (such as
Bauer and Kohler, 2019). We propose a rectified linear unit (ReLU) based deep
neural network (DNN) approach, and contribute to the design of DNN by i)
providing more transparency for practical implementation, ii) defining
different types of sparsity, iii) showing the differentiability, iv) pointing
out the set of effective parameters, and v) offering a new variant of rectified
linear activation function (ReLU), etc. Asymptotic properties are established
accordingly, and a feasible procedure for the purpose of inference is also
proposed. We conduct extensive numerical studies to examine the finite-sample
performance of the estimation methods, and we also evaluate the empirical
relevance and applicability of the proposed models and estimation methods to
real data.
