  Incorporating a human-in-the-loop system when deploying automated decision
support is critical in healthcare contexts to create trust, as well as provide
reliable performance on a patient-to-patient basis. Deep learning methods while
having high performance, do not allow for this patient-centered approach due to
the lack of uncertainty representation. Thus, we present a framework of
uncertainty representation evaluated for medical image segmentation, using
MCU-Net which combines a U-Net with Monte Carlo Dropout, evaluated with four
different uncertainty metrics. The framework augments this by adding a
human-in-the-loop aspect based on an uncertainty threshold for automated
referral of uncertain cases to a medical professional. We demonstrate that
MCU-Net combined with epistemic uncertainty and an uncertainty threshold tuned
for this application maximizes automated performance on an individual patient
level, yet refers truly uncertain cases. This is a step towards uncertainty
representations when deploying machine learning based decision support in
healthcare settings.
