  Recent work linking deep neural networks and dynamical systems opened up new
avenues to analyze deep learning. In particular, it is observed that new
insights can be obtained by recasting deep learning as an optimal control
problem on difference or differential equations. However, the mathematical
aspects of such a formulation have not been systematically explored. This paper
introduces the mathematical formulation of the population risk minimization
problem in deep learning as a mean-field optimal control problem. Mirroring the
development of classical optimal control, we state and prove optimality
conditions of both the Hamilton-Jacobi-Bellman type and the Pontryagin type.
These mean-field results reflect the probabilistic nature of the learning
problem. In addition, by appealing to the mean-field Pontryagin's maximum
principle, we establish some quantitative relationships between population and
empirical learning problems. This serves to establish a mathematical foundation
for investigating the algorithmic and theoretical connections between optimal
control and deep learning.
