  Self-supervised visual representation learning has seen huge progress
recently, but no large scale evaluation has compared the many models now
available. We evaluate the transfer performance of 13 top self-supervised
models on 40 downstream tasks, including many-shot and few-shot recognition,
object detection, and dense prediction. We compare their performance to a
supervised baseline and show that on most tasks the best self-supervised models
outperform supervision, confirming the recently observed trend in the
literature. We find ImageNet Top-1 accuracy to be highly correlated with
transfer to many-shot recognition, but increasingly less so for few-shot,
object detection and dense prediction. No single self-supervised method
dominates overall, suggesting that universal pre-training is still unsolved.
Our analysis of features suggests that top self-supervised learners fail to
preserve colour information as well as supervised alternatives, but tend to
induce better classifier calibration, and less attentive overfitting than
supervised learners.
