  Optimizing over the stationary distribution of stochastic differential
equations (SDEs) is computationally challenging. A new forward propagation
algorithm has been recently proposed for the online optimization of SDEs. The
algorithm solves an SDE, derived using forward differentiation, which provides
a stochastic estimate for the gradient. The algorithm continuously updates the
SDE model's parameters and the gradient estimate simultaneously. This paper
studies the convergence of the forward propagation algorithm for nonlinear
dissipative SDEs. We leverage the ergodicity of this class of nonlinear SDEs to
characterize the convergence rate of the transition semi-group and its
derivatives. Then, we prove bounds on the solution of a Poisson partial
differential equation (PDE) for the expected time integral of the algorithm's
stochastic fluctuations around the direction of steepest descent. We then
re-write the algorithm using the PDE solution, which allows us to characterize
the parameter evolution around the direction of steepest descent. Our main
result is a convergence theorem for the forward propagation algorithm for
nonlinear dissipative SDEs.
