  Continual learning (CL) is a fundamental topic in machine learning, where the
goal is to train a model with continuously incoming data and tasks. Due to the
memory limit, we cannot store all the historical data, and therefore confront
the ``catastrophic forgetting'' problem, i.e., the performance on the previous
tasks can substantially decrease because of the missing information in the
latter period. Though a number of elegant methods have been proposed, the
catastrophic forgetting phenomenon still cannot be well avoided in practice. In
this paper, we study the problem from the gradient perspective, where our aim
is to develop an effective algorithm to calibrate the gradient in each updating
step of the model; namely, our goal is to guide the model to be updated in the
right direction under the situation that a large amount of historical data are
unavailable. Our idea is partly inspired by the seminal stochastic variance
reduction methods (e.g., SVRG and SAGA) for reducing the variance of gradient
estimation in stochastic gradient descent algorithms. Another benefit is that
our approach can be used as a general tool, which is able to be incorporated
with several existing popular CL methods to achieve better performance. We also
conduct a set of experiments on several benchmark datasets to evaluate the
performance in practice.
