  Online continual learning (OCL) aims to continuously learn new data from a
single pass over the online data stream. It generally suffers from the
catastrophic forgetting issue. Existing replay-based methods effectively
alleviate this issue by replaying part of old data in a proxy-based or
contrastive-based replay manner. In this paper, we conduct a comprehensive
analysis of these two replay manners and find they can be complementary.
Inspired by this finding, we propose a novel replay-based method called
proxy-based contrastive replay (PCR), which replaces anchor-to-sample pairs
with anchor-to-proxy pairs in the contrastive-based loss to alleviate the
phenomenon of forgetting. Based on PCR, we further develop a more advanced
method named holistic proxy-based contrastive replay (HPCR), which consists of
three components. The contrastive component conditionally incorporates
anchor-to-sample pairs to PCR, learning more fine-grained semantic information
with a large training batch. The second is a temperature component that
decouples the temperature coefficient into two parts based on their impacts on
the gradient and sets different values for them to learn more novel knowledge.
The third is a distillation component that constrains the learning process to
keep more historical knowledge. Experiments on four datasets consistently
demonstrate the superiority of HPCR over various state-of-the-art methods.
