  We provide a variable metric stochastic approximation theory. In doing so, we
provide a convergence theory for a large class of online variable metric
methods including the recently introduced online versions of the BFGS algorithm
and its limited-memory LBFGS variant. We also discuss the implications of our
results for learning from expert advice.
