  We present a deep recurrent neural network architecture to solve a class of
stochastic optimal control problems described by fully nonlinear Hamilton
Jacobi Bellmanpartial differential equations. Such PDEs arise when one
considers stochastic dynamics characterized by uncertainties that are additive
and control multiplicative. Stochastic models with the aforementioned
characteristics have been used in computational neuroscience, biology, finance
and aerospace systems and provide a more accurate representation of actuation
than models with additive uncertainty. Previous literature has established the
inadequacy of the linear HJB theory and instead rely on a non-linear
Feynman-Kac lemma resulting in a second order forward-backward stochastic
differential equations representation. However, the proposed solutions that use
this representation suffer from compounding errors and computational complexity
leading to lack of scalability. In this paper, we propose a deep learning based
algorithm that leverages the second order Forward-Backward SDE representation
and LSTM based recurrent neural networks to not only solve such Stochastic
Optimal Control problems but also overcome the problems faced by previous
approaches and scales well to high dimensional systems. The resulting control
algorithm is tested on non-linear systems in robotics and biomechanics to
demonstrate feasibility and out-performance against previous methods.
