  Despite the state-of-the-art performance for medical image segmentation, deep
convolutional neural networks (CNNs) have rarely provided uncertainty
estimations regarding their segmentation outputs, e.g., model (epistemic) and
image-based (aleatoric) uncertainties. In this work, we analyze these different
types of uncertainties for CNN-based 2D and 3D medical image segmentation
tasks. We additionally propose a test-time augmentation-based aleatoric
uncertainty to analyze the effect of different transformations of the input
image on the segmentation output. Test-time augmentation has been previously
used to improve segmentation accuracy, yet not been formulated in a consistent
mathematical framework. Hence, we also propose a theoretical formulation of
test-time augmentation, where a distribution of the prediction is estimated by
Monte Carlo simulation with prior distributions of parameters in an image
acquisition model that involves image transformations and noise. We compare and
combine our proposed aleatoric uncertainty with model uncertainty. Experiments
with segmentation of fetal brains and brain tumors from 2D and 3D Magnetic
Resonance Images (MRI) showed that 1) the test-time augmentation-based
aleatoric uncertainty provides a better uncertainty estimation than calculating
the test-time dropout-based model uncertainty alone and helps to reduce
overconfident incorrect predictions, and 2) our test-time augmentation
outperforms a single-prediction baseline and dropout-based multiple
predictions.
