  In this work, we analyze two of the most fundamental algorithms in
geodesically convex optimization: Riemannian gradient descent and (possibly
inexact) Riemannian proximal point. We quantify their rates of convergence and
produce different variants with several trade-offs. Crucially, we show the
iterates naturally stay in a ball around an optimizer, of radius depending on
the initial distance and, in some cases, on the curvature. In contrast, except
for limited cases, previous works bounded the maximum distance between iterates
and an optimizer only by assumption, leading to incomplete analyses and
unquantified rates. We also provide an implementable inexact proximal point
algorithm yielding new results on minmax problems, and we prove several new
useful properties of Riemannian proximal methods: they work when positive
curvature is present, the proximal operator does not move points away from any
optimizer, and we quantify the smoothness of its induced Moreau envelope.
Further, we explore beyond our theory with empirical tests.
