  This paper introduces a novel approach to improving the training stability of
self-supervised learning (SSL) methods by leveraging a non-parametric memory of
seen concepts. The proposed method involves augmenting a neural network with a
memory component to stochastically compare current image views with previously
encountered concepts. Additionally, we introduce stochastic memory blocks to
regularize training and enforce consistency between image views. We extensively
benchmark our method on many vision tasks, such as linear probing, transfer
learning, low-shot classification, and image retrieval on many datasets. The
experimental results consolidate the effectiveness of the proposed approach in
achieving stable SSL training without additional regularizers while learning
highly transferable representations and requiring less computing time and
resources.
