  Despite the empirical successes of self-supervised learning (SSL) methods, it
is unclear what characteristics of their representations lead to high
downstream accuracies. In this work, we characterize properties that SSL
representations should ideally satisfy. Specifically, we prove necessary and
sufficient conditions such that for any task invariant to given data
augmentations, desired probes (e.g., linear or MLP) trained on that
representation attain perfect accuracy. These requirements lead to a unifying
conceptual framework for improving existing SSL methods and deriving new ones.
For contrastive learning, our framework prescribes simple but significant
improvements to previous methods such as using asymmetric projection heads. For
non-contrastive learning, we use our framework to derive a simple and novel
objective. Our resulting SSL algorithms outperform baselines on standard
benchmarks, including SwAV+multicrops on linear probing of ImageNet.
