  Nonlinear acceleration methods are powerful techniques to speed up
fixed-point iterations. However, many acceleration methods require storing a
large number of previous iterates and this can become impractical if
computational resources are limited. In this paper, we propose a nonlinear
Truncated Generalized Conjugate Residual method (nlTGCR) whose goal is to
exploit the symmetry of the Hessian to reduce memory usage. The proposed method
can be interpreted as either an inexact Newton or a quasi-Newton method. We
show that, with the help of global strategies like residual check techniques,
nlTGCR can converge globally for general nonlinear problems and that under mild
conditions, nlTGCR is able to achieve superlinear convergence. We further
analyze the convergence of nlTGCR in a stochastic setting. Numerical results
demonstrate the superiority of nlTGCR when compared with several other
competitive baseline approaches on a few problems. Our code will be available
in the future.
