  Based on the observation that semantic segmentation errors are partially
predictable, we propose a compact formulation using confusion statistics of the
trained classifier to refine (re-estimate) the initial pixel label hypotheses.
The proposed strategy is contingent upon computing the classifier confusion
probabilities for a given dataset and estimating a relevant prior on the object
classes present in the image to be classified. We provide a procedure to
robustly estimate the confusion probabilities and explore multiple prior
definitions. Experiments are shown comparing performances on multiple
challenging datasets using different priors to improve a state-of-the-art
semantic segmentation classifier. This study demonstrates the potential to
significantly improve semantic labeling and motivates future work for reliable
label prior estimation from images.
