  Backward compatibility of model predictions is a desired property when
updating a machine learning driven application. It allows to seamlessly improve
the underlying model without introducing regression bugs. In classification
tasks these bugs occur in the form of negative flips. This means an instance
that was correctly classified by the old model is now classified incorrectly by
the updated model. This has direct negative impact on the user experience of
such systems e.g. a frequently used voice assistant query is suddenly
misclassified. A common reason to update the model is when new training data
becomes available and needs to be incorporated. Simply retraining the model
with the updated data introduces the unwanted negative flips. We study the
problem of regression during data updates and propose Backward Compatible
Weight Interpolation (BCWI). This method interpolates between the weights of
the old and new model and we show in extensive experiments that it reduces
negative flips without sacrificing the improved accuracy of the new model. BCWI
is straight forward to implement and does not increase inference cost. We also
explore the use of importance weighting during interpolation and averaging the
weights of multiple new models in order to further reduce negative flips.
