  We explore task-free continual learning (CL), in which a model is trained to
avoid catastrophic forgetting in the absence of explicit task boundaries or
identities. Among many efforts on task-free CL, a notable family of approaches
are memory-based that store and replay a subset of training examples. However,
the utility of stored seen examples may diminish over time since CL models are
continually updated. Here, we propose Gradient based Memory EDiting (GMED), a
framework for editing stored examples in continuous input space via gradient
updates, in order to create more "challenging" examples for replay. GMED-edited
examples remain similar to their unedited forms, but can yield increased loss
in the upcoming model updates, thereby making the future replays more effective
in overcoming catastrophic forgetting. By construction, GMED can be seamlessly
applied in conjunction with other memory-based CL algorithms to bring further
improvement. Experiments validate the effectiveness of GMED, and our best
method significantly outperforms baselines and previous state-of-the-art on
five out of six datasets. Code can be found at https://github.com/INK-USC/GMED.
