  A dataset is a shred of crucial evidence to describe a task. However, each
data point in the dataset does not have the same potential, as some of the data
points can be more representative or informative than others. This unequal
importance among the data points may have a large impact in rehearsal-based
continual learning, where we store a subset of the training examples (coreset)
to be replayed later to alleviate catastrophic forgetting. In continual
learning, the quality of the samples stored in the coreset directly affects the
model's effectiveness and efficiency. The coreset selection problem becomes
even more important under realistic settings, such as imbalanced continual
learning or noisy data scenarios. To tackle this problem, we propose Online
Coreset Selection (OCS), a simple yet effective method that selects the most
representative and informative coreset at each iteration and trains them in an
online manner. Our proposed method maximizes the model's adaptation to a
current dataset while selecting high-affinity samples to past tasks, which
directly inhibits catastrophic forgetting. We validate the effectiveness of our
coreset selection mechanism over various standard, imbalanced, and noisy
datasets against strong continual learning baselines, demonstrating that it
improves task adaptation and prevents catastrophic forgetting in a
sample-efficient manner.
