  Alternating least squares is the most widely used algorithm for CP tensor
decomposition. However, alternating least squares may exhibit slow or no
convergence, especially when high accuracy is required. An alternative approach
is to regard CP decomposition as a nonlinear least squares problem and employ
Newton-like methods. Direct solution of linear systems involving an
approximated Hessian is generally expensive. However, recent advancements have
shown that use of an implicit representation of the linear system makes these
methods competitive with alternating least squares. We provide the first
parallel implementation of a Gauss-Newton method for CP decomposition, which
iteratively solves linear least squares problems at each Gauss-Newton step. In
particular, we leverage a formulation that employs tensor contractions for
implicit matrix-vector products within the conjugate gradient method. The use
of tensor contractions enables us to employ the Cyclops library for
distributed-memory tensor computations to parallelize the Gauss-Newton approach
with a high-level Python implementation. In addition, we propose a
regularization scheme for Gauss-Newton method to improve convergence properties
without any additional cost. We study the convergence of variants of the
Gauss-Newton method relative to ALS for finding exact CP decompositions as well
as approximate decompositions of real-world tensors. We evaluate the
performance of sequential and parallel versions of both approaches, and study
the parallel scalability on the Stampede2 supercomputer.
