  In this paper, we propose a vector transport-free stochastic variance reduced
gradient (SVRG) method with general retraction for empirical risk minimization
over Riemannian manifold. Existing SVRG methods on manifold usually consider a
specific retraction operation, and involve additional computational costs such
as parallel transport or vector transport. The vector transport-free SVRG with
general retraction we propose in this paper handles general retraction
operations, and do not need additional computational costs mentioned above. As
a result, we name our algorithm S-SVRG, where the first "S" means simple. We
analyze the iteration complexity of S-SVRG for obtaining an
$\epsilon$-stationary point and its local linear convergence by assuming the \L
ojasiewicz inequality, which naturally holds for PCA and holds with high
probability for matrix completion problem. We also incorporate the
Barzilai-Borwein step size and design a very practical S-SVRG-BB method.
Numerical results on PCA and matrix completion problems are reported to
demonstrate the efficiency of our methods.
