  Nonlinear acceleration algorithms improve the performance of iterative
methods, such as gradient descent, using the information contained in past
iterates. However, their efficiency is still not entirely understood even in
the quadratic case. In this paper, we clarify the convergence analysis by
giving general properties that share several classes of nonlinear acceleration:
Anderson acceleration (and variants), quasi-Newton methods (such as Broyden
Type-I or Type-II, SR1, DFP, and BFGS) and Krylov methods (Conjugate Gradient,
MINRES, GMRES). In particular, we propose a generic family of algorithms that
contains all the previous methods and prove its optimal rate of convergence
when minimizing quadratic functions. We also propose multi-secants updates for
the quasi-Newton methods listed above. We provide a Matlab code implementing
the algorithm.
