  Catastrophic forgetting can be a significant problem for institutions that
must delete historic data for privacy reasons. For example, hospitals might not
be able to retain patient data permanently. But neural networks trained on
recent data alone will tend to forget lessons learned on old data. We present a
differentially private continual learning framework based on variational
inference. We estimate the likelihood of past data given the current model
using differentially private generative models of old datasets.
