  This work presents an approach for incrementally updating deep neural network
(DNN) models in a non-stationary environment. DNN models are sensitive to
changes in input data distribution, which limits their application to problem
settings with stationary input datasets. In a non-stationary environment,
updating a DNN model requires parameter re-training or model fine-tuning. We
propose an unsupervised source-free method to update DNN classification models.
The contributions of this work are two-fold. First, we use trainable Gaussian
prototypes to generate representative samples for future iterations; second,
using unsupervised domain adaptation, we incrementally adapt the existing model
using unlabelled data. Unlike existing methods, our approach can update a DNN
model incrementally for non-stationary source and target tasks without storing
past training data. We evaluated our work on incremental sentiment prediction
and incremental disease prediction applications and compared our approach to
state-of-the-art continual learning, domain adaptation, and ensemble learning
methods. Our results show that our approach achieved improved performance
compared to existing incremental learning methods. We observe minimal
forgetting of past knowledge over many iterations, which can help us develop
unsupervised self-learning systems.
