  Preserving maximal information is one of principles of designing
self-supervised learning methodologies. To reach this goal, contrastive
learning adopts an implicit way which is contrasting image pairs. However, we
believe it is not fully optimal to simply use the contrastive estimation for
preservation. Moreover, it is necessary and complemental to introduce an
explicit solution to preserve more information. From this perspective, we
introduce Preservational Learning to reconstruct diverse image contexts in
order to preserve more information in learned representations. Together with
the contrastive loss, we present Preservational Contrastive Representation
Learning (PCRL) for learning self-supervised medical representations. PCRL
provides very competitive results under the pretraining-finetuning protocol,
outperforming both self-supervised and supervised counterparts in 5
classification/segmentation tasks substantially.
