  Continual learning tries to learn new tasks without forgetting previously
learned ones. In reality, most of the existing artificial neural network(ANN)
models fail, while humans do the same by remembering previous works throughout
their life. Although simply storing all past data can alleviate the problem, it
needs large memory and often infeasible in real-world applications where last
data access is limited. We hypothesize that the model that learns to solve each
task continually has some task-specific properties and some task-invariant
characteristics. We propose a hybrid continual learning model that is more
suitable in real case scenarios to address the issues that has a task-invariant
shared variational autoencoder and T task-specific variational autoencoders.
Our model combines generative replay and architectural growth to prevent
catastrophic forgetting. We show our hybrid model effectively avoids forgetting
and achieves state-of-the-art results on visual continual learning benchmarks
such as MNIST, Permuted MNIST(QMNIST), CIFAR100, and miniImageNet datasets. We
discuss results on a few more datasets, such as SVHN, Fashion-MNIST, EMNIST,
and CIFAR10.
