  One notable weakness of current machine learning algorithms is the poor
ability of models to solve new problems without forgetting previously acquired
knowledge. The Continual Learning paradigm has emerged as a protocol to
systematically investigate settings where the model sequentially observes
samples generated by a series of tasks. In this work, we take a task-agnostic
view of continual learning and develop a hierarchical information-theoretic
optimality principle that facilitates a trade-off between learning and
forgetting. We derive this principle from a Bayesian perspective and show its
connections to previous approaches to continual learning. Based on this
principle, we propose a neural network layer, called the
Mixture-of-Variational-Experts layer, that alleviates forgetting by creating a
set of information processing paths through the network which is governed by a
gating policy. Equipped with a diverse and specialized set of parameters, each
path can be regarded as a distinct sub-network that learns to solve tasks. To
improve expert allocation, we introduce diversity objectives, which we evaluate
in additional ablation studies. Importantly, our approach can operate in a
task-agnostic way, i.e., it does not require task-specific knowledge, as is the
case with many existing continual learning algorithms. Due to the general
formulation based on generic utility functions, we can apply this optimality
principle to a large variety of learning problems, including supervised
learning, reinforcement learning, and generative modeling. We demonstrate the
competitive performance of our method on continual reinforcement learning and
variants of the MNIST, CIFAR-10, and CIFAR-100 datasets.
