  We examine a wide class of stochastic approximation algorithms for solving
(stochastic) nonlinear problems on Riemannian manifolds. Such algorithms arise
naturally in the study of Riemannian optimization, game theory and optimal
transport, but their behavior is much less understood compared to the Euclidean
case because of the lack of a global linear structure on the manifold. We
overcome this difficulty by introducing a suitable Fermi coordinate frame which
allows us to map the asymptotic behavior of the Riemannian Robbins-Monro (RRM)
algorithms under study to that of an associated deterministic dynamical system.
In so doing, we provide a general template of almost sure convergence results
that mirrors and extends the existing theory for Euclidean Robbins-Monro
schemes, despite the significant complications that arise due to the curvature
and topology of the underlying manifold. We showcase the flexibility of the
proposed framework by applying it to a range of retraction-based variants of
the popular optimistic / extra-gradient methods for solving minimization
problems and games, and we provide a unified treatment for their convergence.
