  Semantic segmentation is a fundamental computer vision task with a vast
number of applications. State of the art methods increasingly rely on deep
learning models, known to incorrectly estimate uncertainty and being
overconfident in predictions, especially in data not seen during training. This
is particularly problematic in semantic segmentation due to inherent class
imbalance. Popular uncertainty quantification approaches are task-agnostic and
fail to leverage spatial pixel correlations in uncertainty estimates, crucial
in this task. In this work, a novel training methodology specifically designed
for semantic segmentation is presented. Training samples are weighted by
instance-wise uncertainty masks computed by an ensemble. This is shown to
increase performance on minority classes, boost model generalization and
robustness to domain-shift when compared to using the inverse of class
proportions or no class weights at all. This method addresses the challenges of
class imbalance and uncertainty estimation in semantic segmentation,
potentially enhancing model performance and reliability across various
applications.
