  We present a modified limited memory BFGS (L-BFGS) method that converges
globally and linearly for nonconvex objective functions. Its distinguishing
feature is that it turns into L-BFGS if its iterates cluster at a point near
which the objective is strongly convex with Lipschitz gradients, thereby
inheriting the outstanding effectiveness of the classical method. These strong
convergence guarantees are enabled by a novel form of cautious updating, where
it is decided anew in each iteration which of the stored vectors are used for
updating and which ones are skipped. Moreover, the convergence results also
hold for memory size zero, yielding a new globalization of the Barzilai-Borwein
method with strong convergence guarantees. The convergence analysis is carried
out in infinite dimensional Hilbert space and also strengthens existing
convergence results for L-BFGS-type methods, including for L-BFGS. In
particular, we show that the Wolfe-Powell conditions already imply global
convergence if the gradient of the objective is continuous. This is a finding
of interest in its own right as it is largely independent of the underlying
optimization method. We illustrate the results in numerical experiments.
Additionally, these experiments indicate that if the parameters of the cautious
updating are suitably chosen, the modified method agrees entirely with L-BFGS,
suggesting that L-BFGS is inherently cautious. An implementation of the method
is available on arXiv.
