  Continual learning is the problem of integrating new information in a model
while retaining the knowledge acquired in the past. Despite the tangible
improvements achieved in recent years, the problem of continual learning is
still an open one. A better understanding of the mechanisms behind the
successes and failures of existing continual learning algorithms can unlock the
development of new successful strategies. In this work, we view continual
learning from the perspective of the multi-task loss approximation, and we
compare two alternative strategies, namely local and global approximations. We
classify existing continual learning algorithms based on the approximation
used, and we assess the practical effects of this distinction in common
continual learning settings.Additionally, we study optimal continual learning
objectives in the case of local polynomial approximations and we provide
examples of existing algorithms implementing the optimal objectives
