  A new algorithm is presented for computing a canonical rank-R tensor
approximation that has minimal distance to a given tensor in the Frobenius
norm, where the canonical rank-R tensor consists of the sum of R rank-one
components. Each iteration of the method consists of three steps. In the first
step, a tentative new iterate is generated by a stand-alone one-step process,
for which we use alternating least squares (ALS). In the second step, an
accelerated iterate is generated by a nonlinear generalized minimal residual
(GMRES) approach, recombining previous iterates in an optimal way, and
essentially using the stand-alone one-step process as a preconditioner. In
particular, the nonlinear extension of GMRES is used that was proposed by
Washio and Oosterlee in [ETNA Vol. 15 (2003), pp. 165-185] for nonlinear
partial differential equation problems. In the third step, a line search is
performed for globalization. The resulting nonlinear GMRES (N-GMRES)
optimization algorithm is applied to dense and sparse tensor decomposition test
problems. The numerical tests show that ALS accelerated by N-GMRES may
significantly outperform both stand-alone ALS and a standard nonlinear
conjugate gradient optimization method, especially when highly accurate
stationary points are desired for difficult problems. The proposed N-GMRES
optimization algorithm is based on general concepts and may be applied to other
nonlinear optimization problems.
