  The ability to continuously expand knowledge over time and utilize it to
rapidly generalize to new tasks is a key feature of human linguistic
intelligence. Existing models that pursue rapid generalization to new tasks
(e.g., few-shot learning methods), however, are mostly trained in a single shot
on fixed datasets, unable to dynamically expand their knowledge; while
continual learning algorithms are not specifically designed for rapid
generalization. We present a new learning setup, Continual Learning of Few-Shot
Learners (CLIF), to address the challenges of both learning settings in a
unified setup. CLIF assumes a model learns from a sequence of diverse NLP tasks
arriving sequentially, accumulating knowledge for improved generalization to
new tasks, while also retaining performance on the tasks learned earlier. We
examine how the generalization ability is affected in the continual learning
setup, evaluate a number of continual learning algorithms, and propose a novel
regularized adapter generation approach. We find that catastrophic forgetting
affects generalization ability to a less degree than performance on seen tasks;
while continual learning algorithms can still bring considerable benefit to the
generalization ability.
