  While recent computer vision algorithms achieve impressive performance on
many benchmarks, they lack robustness - presented with an image from a
different distribution, (e.g. weather or lighting conditions not considered
during training), they may produce an erroneous prediction. Therefore, it is
desired that such a model will be able to reliably predict its confidence
measure. In this work, uncertainty estimation for the task of semantic
segmentation is evaluated under a varying level of domain shift: in a
cross-dataset setting and when adapting a model trained on data from the
simulation. It was shown that simple color transformations already provide a
strong baseline, comparable to using more sophisticated style-transfer data
augmentation. Further, by constructing an ensemble consisting of models using
different backbones and/or augmentation methods, it was possible to improve
significantly model performance in terms of overall accuracy and uncertainty
estimation under the domain shift setting. The Expected Calibration Error (ECE)
on challenging GTA to Cityscapes adaptation was reduced from 4.05 to the
competitive value of 1.1. Further, an ensemble of models was utilized in the
self-training setting to improve the pseudo-labels generation, which resulted
in a significant gain in the final model accuracy, compared to the standard
fine-tuning (without ensemble).
