  This paper introduces a hypothesis space for deep learning that employs deep
neural networks (DNNs). By treating a DNN as a function of two variables, the
physical variable and parameter variable, we consider the primitive set of the
DNNs for the parameter variable located in a set of the weight matrices and
biases determined by a prescribed depth and widths of the DNNs. We then
complete the linear span of the primitive DNN set in a weak* topology to
construct a Banach space of functions of the physical variable. We prove that
the Banach space so constructed is a reproducing kernel Banach space (RKBS) and
construct its reproducing kernel. We investigate two learning models,
regularized learning and minimum interpolation problem in the resulting RKBS,
by establishing representer theorems for solutions of the learning models. The
representer theorems unfold that solutions of these learning models can be
expressed as linear combination of a finite number of kernel sessions
determined by given data and the reproducing kernel.
