  Machine learning models are not static and may need to be retrained on
slightly changed datasets, for instance, with the addition or deletion of a set
of data points. This has many applications, including privacy, robustness, bias
reduction, and uncertainty quantifcation. However, it is expensive to retrain
models from scratch. To address this problem, we propose the DeltaGrad
algorithm for rapid retraining machine learning models based on information
cached during the training phase. We provide both theoretical and empirical
support for the effectiveness of DeltaGrad, and show that it compares favorably
to the state of the art.
