  Rehearsal approaches enjoy immense popularity with Continual Learning (CL)
practitioners. These methods collect samples from previously encountered data
distributions in a small memory buffer; subsequently, they repeatedly optimize
on the latter to prevent catastrophic forgetting. This work draws attention to
a hidden pitfall of this widespread practice: repeated optimization on a small
pool of data inevitably leads to tight and unstable decision boundaries, which
are a major hindrance to generalization. To address this issue, we propose
Lipschitz-DrivEn Rehearsal (LiDER), a surrogate objective that induces
smoothness in the backbone network by constraining its layer-wise Lipschitz
constants w.r.t. replay examples. By means of extensive experiments, we show
that applying LiDER delivers a stable performance gain to several
state-of-the-art rehearsal CL methods across multiple datasets, both in the
presence and absence of pre-training. Through additional ablative experiments,
we highlight peculiar aspects of buffer overfitting in CL and better
characterize the effect produced by LiDER. Code is available at
https://github.com/aimagelab/LiDER
