  Supervised machine learning provides state-of-the-art solutions to a wide
range of computer vision problems. However, the need for copious labelled
training data limits the capabilities of these algorithms in scenarios where
such input is scarce or expensive. Self-supervised learning offers a way to
lower the need for manually annotated data by pretraining models for a specific
domain on unlabelled data. In this approach, labelled data are solely required
to fine-tune models for downstream tasks. Medical image segmentation is a field
where labelling data requires expert knowledge and collecting large labelled
datasets is challenging; therefore, self-supervised learning algorithms promise
substantial improvements in this field. Despite this, self-supervised learning
algorithms are used rarely to pretrain medical image segmentation networks. In
this paper, we elaborate and analyse the effectiveness of supervised and
self-supervised pretraining approaches on downstream medical image
segmentation, focusing on convergence and data efficiency. We find that
self-supervised pretraining on natural images and target-domain-specific images
leads to the fastest and most stable downstream convergence. In our experiments
on the ACDC cardiac segmentation dataset, this pretraining approach achieves
4-5 times faster fine-tuning convergence compared to an ImageNet pretrained
model. We also show that this approach requires less than five epochs of
pretraining on domain-specific data to achieve such improvement in the
downstream convergence time. Finally, we find that, in low-data scenarios,
supervised ImageNet pretraining achieves the best accuracy, requiring less than
100 annotated samples to realise close to minimal error.
