  Momentum methods play a significant role in optimization. Examples include
Nesterov's accelerated gradient method and the conditional gradient algorithm.
Several momentum methods are provably optimal under standard oracle models, and
all use a technique called estimate sequences to analyze their convergence
properties. The technique of estimate sequences has long been considered
difficult to understand, leading many researchers to generate alternative,
"more intuitive" methods and analyses. We show there is an equivalence between
the technique of estimate sequences and a family of Lyapunov functions in both
continuous and discrete time. This connection allows us to develop a simple and
unified analysis of many existing momentum algorithms, introduce several new
algorithms, and strengthen the connection between algorithms and
continuous-time dynamical systems.
