  Continual Learning is a learning paradigm where learning systems are trained
with sequential or streaming tasks. Two notable directions among the recent
advances in continual learning with neural networks are ($i$) variational Bayes
based regularization by learning priors from previous tasks, and, ($ii$)
learning the structure of deep networks to adapt to new tasks. So far, these
two approaches have been orthogonal. We present a novel Bayesian approach to
continual learning based on learning the structure of deep neural networks,
addressing the shortcomings of both these approaches. The proposed model learns
the deep structure for each task by learning which weights to be used, and
supports inter-task transfer through the overlapping of different sparse
subsets of weights learned by different tasks. Experimental results on
supervised and unsupervised benchmarks shows that our model performs comparably
or better than recent advances in continual learning setting.
