  With the purpose of examining biased updates in variance-reduced stochastic
gradient methods, we introduce SVAG, a SAG/SAGA-like method with adjustable
bias. SVAG is analyzed in a cocoercive root-finding setting, a setting which
yields the same results as in the usual smooth convex optimization setting for
the ordinary proximal-gradient method. We show that the same is not true for
SVAG when biased updates are used. The step-size requirements for when the
operators are gradients are significantly less restrictive compared to when
they are not. This highlights the need to not rely solely on cocoercivity when
analyzing variance-reduced methods meant for optimization. Our analysis either
match or improve on previously known convergence conditions for SAG and SAGA.
However, in the biased cases they still do not correspond well with practical
experiences and we therefore examine the effect of bias numerically on a set of
classification problems. The choice of bias seem to primarily affect the early
stages of convergence and in most cases the differences vanish in the later
stages of convergence. However, the effect of the bias choice is still
significant in a couple of cases.
