  In this paper, we construct approximated solutions of Differential Equations
(DEs) using the Deep Neural Network (DNN). Furthermore, we present an
architecture that includes the process of finding model parameters through
experimental data, the inverse problem. That is, we provide a unified framework
of DNN architecture that approximates an analytic solution and its model
parameters simultaneously. The architecture consists of a feed forward DNN with
non-linear activation functions depending on DEs, automatic differentiation,
reduction of order, and gradient based optimization method. We also prove
theoretically that the proposed DNN solution converges to an analytic solution
in a suitable function space for fundamental DEs. Finally, we perform numerical
experiments to validate the robustness of our simplistic DNN architecture for
1D transport equation, 2D heat equation, 2D wave equation, and the
Lotka-Volterra system.
