  Gradient-based meta-learning has proven to be highly effective at learning
model initializations, representations, and update rules that allow fast
adaptation from a few samples. The core idea behind these approaches is to use
fast adaptation and generalization -- two second-order metrics -- as training
signals on a meta-training dataset. However, little attention has been given to
other possible second-order metrics. In this paper, we investigate a different
training signal -- robustness to catastrophic interference -- and demonstrate
that representations learned by directing minimizing interference are more
conducive to incremental learning than those learned by just maximizing fast
adaptation.
