  We propose incremental (re)training of a neural network model to cope with a
continuous flow of new data in inference during model serving. As such, this is
a life-long learning process. We address two challenges of life-long
retraining: catastrophic forgetting and efficient retraining. If we combine all
past and new data it can easily become intractable to retrain the neural
network model. On the other hand, if the model is retrained using only new
data, it can easily suffer catastrophic forgetting and thus it is paramount to
strike the right balance. Moreover, if we retrain all weights of the model
every time new data is collected, retraining tends to require too many
computing resources. To solve these two issues, we propose a novel retraining
model that can select important samples and important weights utilizing
multi-armed bandits. To further address forgetting, we propose a new
regularization term focusing on synapse and neuron importance. We analyze
multiple datasets to document the outcome of the proposed retraining methods.
Various experiments demonstrate that our retraining methodologies mitigate the
catastrophic forgetting problem while boosting model performance.
