  Continual learning aims to learn continuously from a stream of tasks and data
in an online-learning fashion, being capable of exploiting what was learned
previously to improve current and future tasks while still being able to
perform well on the previous tasks. One common limitation of many existing
continual learning methods is that they often train a model directly on all
available training data without validation due to the nature of continual
learning, thus suffering poor generalization at test time. In this work, we
present a novel framework of continual learning named "Bilevel Continual
Learning" (BCL) by unifying a {\it bilevel optimization} objective and a {\it
dual memory management} strategy comprising both episodic memory and
generalization memory to achieve effective knowledge transfer to future tasks
and alleviate catastrophic forgetting on old tasks simultaneously. Our
extensive experiments on continual learning benchmarks demonstrate the efficacy
of the proposed BCL compared to many state-of-the-art methods. Our
implementation is available at
https://github.com/phquang/bilevel-continual-learning.
