  Stochastic variance reduced optimization methods are known to be globally
convergent while they suffer from slow local convergence, especially when
moderate or high accuracy is needed. To alleviate this problem, we propose an
optimization algorithm -- which we refer to as a hybrid acceleration scheme --
for a class of proximal variance reduced stochastic optimization algorithms.
The proposed optimization scheme combines a fast locally convergent algorithm,
such as a quasi--Newton method, with a globally convergent variance reduced
stochastic algorithm, for instance SAGA or L--SVRG. Our global convergence
result of the hybrid acceleration method is based on specific safeguard
conditions that need to be satisfied for a step of the locally fast convergent
method to be accepted.
  We prove that the sequence of the iterates generated by the hybrid
acceleration scheme converges almost surely to a solution of the underlying
optimization problem. We also provide numerical experiments that show
significantly improved convergence of the hybrid acceleration scheme compared
to the basic stochastic variance reduced optimization algorithm.
