  In the past years, deep learning has seen an increase in usage in the domain
of histopathological applications. However, while these approaches have shown
great potential, in high-risk environments deep learning models need to be able
to judge their uncertainty and be able to reject inputs when there is a
significant chance of misclassification. In this work, we conduct a rigorous
evaluation of the most commonly used uncertainty and robustness methods for the
classification of Whole Slide Images, with a focus on the task of selective
classification, where the model should reject the classification in situations
in which it is uncertain. We conduct our experiments on tile-level under the
aspects of domain shift and label noise, as well as on slide-level. In our
experiments, we compare Deep Ensembles, Monte-Carlo Dropout, Stochastic
Variational Inference, Test-Time Data Augmentation as well as ensembles of the
latter approaches. We observe that ensembles of methods generally lead to
better uncertainty estimates as well as an increased robustness towards domain
shifts and label noise, while contrary to results from classical computer
vision benchmarks no systematic gain of the other methods can be shown. Across
methods, a rejection of the most uncertain samples reliably leads to a
significant increase in classification accuracy on both in-distribution as well
as out-of-distribution data. Furthermore, we conduct experiments comparing
these methods under varying conditions of label noise. Lastly, we publish our
code framework to facilitate further research on uncertainty estimation on
histopathological data.
