  Recently, optimization on the Riemannian manifold has provided new insights
to the optimization community. In this regard, the manifold taken as the
probability measure metric space equipped with the second-order Wasserstein
distance is of particular interest, since optimization on it can be linked to
practical sampling processes. In general, the standard (continuous)
optimization method on Wasserstein space is Riemannian gradient flow (i.e.,
Langevin dynamics when minimizing KL divergence). In this paper, we aim to
enrich the continuous optimization methods in the Wasserstein space, by
extending the gradient flow on it into the stochastic gradient descent (SGD)
flow and stochastic variance reduction gradient (SVRG) flow. The two flows in
Euclidean space are standard continuous stochastic methods, while their
Riemannian counterparts are unexplored. By leveraging the property of
Wasserstein space, we construct stochastic differential equations (SDEs) to
approximate the corresponding discrete dynamics of desired Riemannian
stochastic methods in Euclidean space. Then, our probability measures flows are
obtained by the Fokker-Planck equation. Finally, the convergence rates of our
Riemannian stochastic flows are proven, which match the results in Euclidean
space.
