  This work proposes a model for continual learning on tasks involving temporal
sequences, specifically, human motions. It improves on a recently proposed
brain-inspired replay model (BI-R) by building a biologically-inspired
conditional temporal variational autoencoder (BI-CTVAE), which instantiates a
latent mixture-of-Gaussians for class representation. We investigate a novel
continual-learning-to-generate (CL2Gen) scenario where the model generates
motion sequences of different classes. The generative accuracy of the model is
tested over a set of tasks. The final classification accuracy of BI-CTVAE on a
human motion dataset after sequentially learning all action classes is 78%,
which is 63% higher than using no-replay, and only 5.4% lower than a
state-of-the-art offline trained GRU model.
