  We propose new and original mathematical connections between Hamilton-Jacobi
(HJ) partial differential equations (PDEs) with initial data and neural network
architectures. Specifically, we prove that some classes of neural networks
correspond to representation formulas of HJ PDE solutions whose Hamiltonians
and initial data are obtained from the parameters of the neural networks. These
results do not rely on universal approximation properties of neural networks;
rather, our results show that some classes of neural network architectures
naturally encode the physics contained in some HJ PDEs. Our results naturally
yield efficient neural network-based methods for evaluating solutions of some
HJ PDEs in high dimension without using grids or numerical approximations. We
also present some numerical results for solving some inverse problems involving
HJ PDEs using our proposed architectures.
