  In this paper, we introduce various mechanisms to obtain accelerated
first-order stochastic optimization algorithms when the objective function is
convex or strongly convex. Specifically, we extend the Catalyst approach
originally designed for deterministic objectives to the stochastic setting.
Given an optimization method with mild convergence guarantees for strongly
convex problems, the challenge is to accelerate convergence to a
noise-dominated region, and then achieve convergence with an optimal worst-case
complexity depending on the noise variance of the gradients. A side
contribution of our work is also a generic analysis that can handle inexact
proximal operators, providing new insights about the robustness of stochastic
algorithms when the proximal operator cannot be exactly computed.
