  We present a novel algorithm for learning the parameters of hidden Markov
models (HMMs) in a geometric setting where the observations take values in
Riemannian manifolds. In particular, we elevate a recent second-order method of
moments algorithm that incorporates non-consecutive correlations to a more
general setting where observations take place in a Riemannian symmetric space
of non-positive curvature and the observation likelihoods are Riemannian
Gaussians. The resulting algorithm decouples into a Riemannian Gaussian mixture
model estimation algorithm followed by a sequence of convex optimization
procedures. We demonstrate through examples that the learner can result in
significantly improved speed and numerical accuracy compared to existing
learners.
