  Self-supervised learning (SSL) has emerged as an effective paradigm for
deriving general representations from vast amounts of unlabeled data. However,
as real-world applications continually integrate new content, the high
computational and resource demands of SSL necessitate continual learning rather
than complete retraining. This poses a challenge in striking a balance between
stability and plasticity when adapting to new information. In this paper, we
employ Centered Kernel Alignment for quantitatively analyzing model stability
and plasticity, revealing the critical roles of batch normalization layers for
stability and convolutional layers for plasticity. Motivated by this, we
propose Branch-tuning, an efficient and straightforward method that achieves a
balance between stability and plasticity in continual SSL. Branch-tuning
consists of branch expansion and compression, and can be easily applied to
various SSL methods without the need of modifying the original methods,
retaining old data or models. We validate our method through incremental
experiments on various benchmark datasets, demonstrating its effectiveness and
practical value in real-world scenarios. We hope our work offers new insights
for future continual self-supervised learning research. The code will be made
publicly available.
