  A modified Deep BSDE (backward differential equation) learning method with
measurability loss, called Deep BSDE-ML method, is introduced in this paper to
solve a kind of linear decoupled forward-backward stochastic differential
equations (FBSDEs), which is encountered in the policy evaluation of learning
the optimal feedback policies of a class of stochastic control problems. The
measurability loss is characterized via the measurability of BSDE's state at
the forward initial time, which differs from that related to terminal state of
the known Deep BSDE method. Though the minima of the two loss functions are
shown to be equal, this measurability loss is proved to be equal to the
expected mean squared error between the true diffusion term of BSDE and its
approximation. This crucial observation extends the application of the Deep
BSDE method -- approximating the gradients of the solution of a partial
differential equation (PDE) instead of the solution itself. Simultaneously, a
learning-based framework is introduced to search an optimal feedback control of
a deterministic nonlinear system. Specifically, by introducing Gaussian
exploration noise, we are aiming to learn a robust optimal controller under
this stochastic case. This reformulation sacrifices the optimality to some
extent, but as suggested in reinforcement learning (RL) exploration noise is
essential to enable the model-free learning.
