  Confidence curves are used in uncertainty validation to assess how large
uncertainties ($u_{E}$) are associated with large errors ($E$). An oracle curve
is commonly used as reference to estimate the quality of the tested datasets.
The oracle is a perfect, deterministic, error predictor, such as $|E|=\pm
u_{E}$, which corresponds to a very unlikely error distribution in a
probabilistic framework and is unable unable to inform us on the calibration of
$u_{E}$. I propose here to replace the oracle by a probabilistic reference
curve, deriving from the more realistic scenario where errors should be random
draws from a distribution with standard deviation $u_{E}$. The probabilistic
curve and its confidence interval enable a direct test of the quality of a
confidence curve. Paired with the probabilistic reference, a confidence curve
can be used to check the calibration and tightness of prediction uncertainties.
