  Recent work studies the supervised online continual learning setting where a
learner receives a stream of data whose class distribution changes over time.
Distinct from other continual learning settings the learner is presented new
samples only once and must distinguish between all seen classes. A number of
successful methods in this setting focus on storing and replaying a subset of
samples alongside incoming data in a computationally efficient manner. One
recent proposal ER-AML achieved strong performance in this setting by applying
an asymmetric loss based on contrastive learning to the incoming data and
replayed data. However, a key ingredient of the proposed method is avoiding
contrasts between incoming data and stored data, which makes it impractical for
the setting where only one new class is introduced in each phase of the stream.
In this work we adapt a recently proposed approach (\textit{BYOL}) from
self-supervised learning to the supervised learning setting, unlocking the
constraint on contrasts. We then show that supplementing this with additional
regularization on class prototypes yields a new method that achieves strong
performance in the one-class incremental learning setting and is competitive
with the top performing methods in the multi-class incremental setting.
