  Unsupervised learning has been a long-standing goal of machine learning and
is especially important for medical image analysis, where the learning can
compensate for the scarcity of labeled datasets. A promising subclass of
unsupervised learning is self-supervised learning, which aims to learn salient
features using the raw input as the learning signal. In this paper, we use a
contrastive self-supervised learning method called SimCLR that achieved
state-of-the-art results on natural-scene images and apply this method to
digital histopathology by collecting and pretraining on 57 histopathology
datasets without any labels. We find that combining multiple multi-organ
datasets with different types of staining and resolution properties improves
the quality of the learned features. Furthermore, we find using more images for
pretraining leads to a better performance in multiple downstream tasks. Linear
classifiers trained on top of the learned features show that networks
pretrained on digital histopathology datasets perform better than ImageNet
pretrained networks, boosting task performances by more than 28% in F1 scores
on average. These findings may also be useful when applying newer contrastive
techniques to histopathology data. Pretrained PyTorch models are made publicly
available at https://github.com/ozanciga/self-supervised-histopathology.
