  This paper considers continual learning of large-scale pretrained neural
machine translation model without accessing the previous training data or
introducing model separation. We argue that the widely used
regularization-based methods, which perform multi-objective learning with an
auxiliary loss, suffer from the misestimate problem and cannot always achieve a
good balance between the previous and new tasks. To solve the problem, we
propose a two-stage training method based on the local features of the real
loss. We first search low forgetting risk regions, where the model can retain
the performance on the previous task as the parameters are updated, to avoid
the catastrophic forgetting problem. Then we can continually train the model
within this region only with the new training data to fit the new task.
Specifically, we propose two methods to search the low forgetting risk regions,
which are based on the curvature of loss and the impacts of the parameters on
the model output, respectively. We conduct experiments on domain adaptation and
more challenging language adaptation tasks, and the experimental results show
that our method can achieve significant improvements compared with several
strong baselines.
