  In this work, we observe that many existing self-supervised learning
algorithms can be both unified and generalized when seen through the lens of
equivariant representations. Specifically, we introduce a general framework we
call Homomorphic Self-Supervised Learning, and theoretically show how it may
subsume the use of input-augmentations provided an augmentation-homomorphic
feature extractor. We validate this theory experimentally for simple
augmentations, demonstrate how the framework fails when representational
structure is removed, and further empirically explore how the parameters of
this framework relate to those of traditional augmentation-based
self-supervised learning. We conclude with a discussion of the potential
benefits afforded by this new perspective on self-supervised learning.
