  This work is on constrained large-scale non-convex optimization where the
constraint set implies a manifold structure. Solving such problems is important
in a multitude of fundamental machine learning tasks. Recent advances on
Riemannian optimization have enabled the convenient recovery of solutions by
adapting unconstrained optimization algorithms over manifolds. However, it
remains challenging to scale up and meanwhile maintain stable convergence rates
and handle saddle points. We propose a new second-order Riemannian optimization
algorithm, aiming at improving convergence rate and reducing computational
cost. It enhances the Riemannian trust-region algorithm that explores curvature
information to escape saddle points through a mixture of subsampling and cubic
regularization techniques. We conduct rigorous analysis to study the
convergence behavior of the proposed algorithm. We also perform extensive
experiments to evaluate it based on two general machine learning tasks using
multiple datasets. The proposed algorithm exhibits improved computational speed
and convergence behavior compared to a large set of state-of-the-art Riemannian
optimization algorithms.
