  Grading precancerous lesions on whole slide images is a challenging task: the
continuous space of morphological phenotypes makes clear-cut decisions between
different grades often difficult, leading to low inter- and intra-rater
agreements. More and more Artificial Intelligence (AI) algorithms are developed
to help pathologists perform and standardize their diagnosis. However, those
models can render their prediction without consideration of the ambiguity of
the classes and can fail without notice which prevent their wider acceptance in
a clinical context. In this paper, we propose a new score to measure the
confidence of AI models in grading tasks. Our confidence score is specifically
adapted to ordinal output variables, is versatile and does not require extra
training or additional inferences nor particular architecture changes.
Comparison to other popular techniques such as Monte Carlo Dropout and deep
ensembles shows that our method provides state-of-the art results, while being
simpler, more versatile and less computationally intensive. The score is also
easily interpretable and consistent with real life hesitations of pathologists.
We show that the score is capable of accurately identifying mispredicted slides
and that accuracy for high confidence decisions is significantly higher than
for low-confidence decisions (gap in AUC of 17.1% on the test set). We believe
that the proposed confidence score could be leveraged by pathologists directly
in their workflow and assist them on difficult tasks such as grading
precancerous lesions.
