  We present a neural network approach for approximating the value function of
high-dimensional stochastic control problems. Our training process
simultaneously updates our value function estimate and identifies the part of
the state space likely to be visited by optimal trajectories. Our approach
leverages insights from optimal control theory and the fundamental relation
between semi-linear parabolic partial differential equations and
forward-backward stochastic differential equations. To focus the sampling on
relevant states during neural network training, we use the stochastic
Pontryagin maximum principle (PMP) to obtain the optimal controls for the
current value function estimate. By design, our approach coincides with the
method of characteristics for the non-viscous Hamilton-Jacobi-Bellman equation
arising in deterministic control problems. Our training loss consists of a
weighted sum of the objective functional of the control problem and penalty
terms that enforce the HJB equations along the sampled trajectories.
Importantly, training is unsupervised in that it does not require solutions of
the control problem.
  Our numerical experiments highlight our scheme's ability to identify the
relevant parts of the state space and produce meaningful value estimates. Using
a two-dimensional model problem, we demonstrate the importance of the
stochastic PMP to inform the sampling and compare to a finite element approach.
With a nonlinear control affine quadcopter example, we illustrate that our
approach can handle complicated dynamics. For a 100-dimensional benchmark
problem, we demonstrate that our approach improves accuracy and
time-to-solution and, via a modification, we show the wider applicability of
our scheme.
