  We present Extended Riemannian Stochastic Derivative-Free Optimization
(Extended RSDFO), a novel population-based stochastic optimization algorithm on
Riemannian manifolds that addresses the locality and implicit assumptions of
manifold optimization in the literature.
  We begin by investigating the Information Geometrical structure of
statistical model over Riemannian manifolds. This establishes a geometrical
framework of Extended RSDFO using both the statistical geometry of the decision
space and the Riemannian geometry of the search space. We construct locally
inherited probability distribution via an orientation-preserving diffeomorphic
bundle morphism, and then extend the information geometrical structure to
mixture densities over totally bounded subsets of manifolds. The former relates
the information geometry of the decision space and the local point estimations
on the search space manifold. The latter overcomes the locality of parametric
probability distributions on Riemannian manifolds.
  We then construct Extended RSDFO and study its structure and properties from
a geometrical perspective. We show that Extended RSDFO's expected fitness
improves monotonically and it's global eventual convergence in finitely many
steps on connected compact Riemannian manifolds.
  Extended RSDFO is compared to state-of-the-art manifold optimization
algorithms on multi-modal optimization problems over a variety of manifolds.
  In particular, we perform a novel synthetic experiment on Jacob's ladder to
motivate and necessitate manifold optimization. Jacob's ladder is a non-compact
manifold of countably infinite genus, which cannot be expressed as polynomial
constraints and does not have a global representation in an ambient Euclidean
space. Optimization problems on Jacob's ladder thus cannot be addressed by
traditional (constraint) optimization methods on Euclidean spaces.
