  Uncertainty estimation plays an important role for future reliable deployment
of deep segmentation models in safety-critical scenarios such as medical
applications. However, existing methods for uncertainty estimation have been
limited by the lack of explicit guidance for calibrating the prediction risk
and model confidence. In this work, we propose a novel fine-grained reward
maximization (FGRM) framework, to address uncertainty estimation by directly
utilizing an uncertainty metric related reward function with a reinforcement
learning based model tuning algorithm. This would benefit the model uncertainty
estimation through direct optimization guidance for model calibration.
Specifically, our method designs a new uncertainty estimation reward function
using the calibration metric, which is maximized to fine-tune an evidential
learning pre-trained segmentation model for calibrating prediction risk.
Importantly, we innovate an effective fine-grained parameter update scheme,
which imposes fine-grained reward-weighting of each network parameter according
to the parameter importance quantified by the fisher information matrix. To the
best of our knowledge, this is the first work exploring reward optimization for
model uncertainty estimation in safety-critical vision tasks. The effectiveness
of our method is demonstrated on two large safety-critical surgical scene
segmentation datasets under two different uncertainty estimation settings. With
real-time one forward pass at inference, our method outperforms
state-of-the-art methods by a clear margin on all the calibration metrics of
uncertainty estimation, while maintaining a high task accuracy for the
segmentation results. Code is available at
\url{https://github.com/med-air/FGRM}.
