  Self-supervised representation learning has achieved impressive results in
recent years, with experiments primarily coming on ImageNet or other similarly
large internet imagery datasets. There has been little to no work with these
methods on other smaller domains, such as satellite, textural, or biological
imagery. We experiment with several popular methods on an unprecedented variety
of domains. We discover, among other findings, that Rotation is by far the most
semantically meaningful task, with much of the performance of Jigsaw and
Instance Discrimination being attributable to the nature of their induced
distribution rather than semantic understanding. Additionally, there are
several areas, such as fine-grain classification, where all tasks underperform.
We quantitatively and qualitatively diagnose the reasons for these failures and
successes via novel experiments studying pretext generalization, random
labelings, and implicit dimensionality. Code and models are available at
https://github.com/BramSW/Extending_SSRL_Across_Domains/.
