  We consider the problem of learning multiple tasks in a continual learning
setting in which data from different tasks is presented to the learner in a
streaming fashion. A key challenge in this setting is the so-called
"catastrophic forgetting problem", in which the performance of the learner in
an "old task" decreases when subsequently trained on a "new task". Existing
continual learning methods, such as Averaged Gradient Episodic Memory (A-GEM)
and Orthogonal Gradient Descent (OGD), address catastrophic forgetting by
minimizing the loss for the current task without increasing the loss for
previous tasks. However, these methods assume the learner knows when the task
changes, which is unrealistic in practice. In this paper, we alleviate the need
to provide the algorithm with information about task changes by using an online
clustering-based approach on a dynamically updated finite pool of samples or
gradients. We thereby successfully counteract catastrophic forgetting in one of
the hardest settings, namely: domain-incremental learning, a setting for which
the problem was previously unsolved. We showcase the benefits of our approach
by applying these ideas to projection-based methods, such as A-GEM and OGD,
which lead to task-agnostic versions of them. Experiments on real datasets
demonstrate the effectiveness of the proposed strategy and its promising
performance compared to state-of-the-art methods.
