  Online continual learning aims to learn from a non-IID stream of data from a
number of different tasks, where the learner is only allowed to consider data
once. Methods are typically allowed to use a limited buffer to store some of
the images in the stream. Recently, it was found that feature replay, where an
intermediate layer representation of the image is stored (or generated) leads
to superior results than image replay, while requiring less memory. Quantized
exemplars can further reduce the memory usage. However, a drawback of these
methods is that they use a fixed (or very intransigent) backbone network. This
significantly limits the learning of representations that can discriminate
between all tasks. To address this problem, we propose an auxiliary classifier
auto-encoder (ACAE) module for feature replay at intermediate layers with high
compression rates. The reduced memory footprint per image allows us to save
more exemplars for replay. In our experiments, we conduct task-agnostic
evaluation under online continual learning setting and get state-of-the-art
performance on ImageNet-Subset, CIFAR100 and CIFAR10 dataset.
