  Accurate uncertainty estimation is a critical need for the medical imaging
community. A variety of methods have been proposed, all direct extensions of
classification uncertainty estimations techniques. The independent pixel-wise
uncertainty estimates, often based on the probabilistic interpretation of
neural networks, do not take into account anatomical prior knowledge and
consequently provide sub-optimal results to many segmentation tasks. For this
reason, we propose CRISP a ContRastive Image Segmentation for uncertainty
Prediction method. At its core, CRISP implements a contrastive method to learn
a joint latent space which encodes a distribution of valid segmentations and
their corresponding images. We use this joint latent space to compare
predictions to thousands of latent vectors and provide anatomically consistent
uncertainty maps. Comprehensive studies performed on four medical image
databases involving different modalities and organs underlines the superiority
of our method compared to state-of-the-art approaches.
