  Modern semantic segmentation methods devote much effect to adjusting image
feature representations to improve the segmentation performance in various
ways, such as architecture design, attention mechnism, etc. However, almost all
those methods neglect the particularity of class weights (in the classification
layer) in segmentation models. In this paper, we notice that the class weights
of categories that tend to share many adjacent boundary pixels lack
discrimination, thereby limiting the performance. We call this issue
Boundary-caused Class Weights Confusion (BCWC). We try to focus on this problem
and propose a novel method named Embedded Conditional Random Field (E-CRF) to
alleviate it. E-CRF innovatively fuses the CRF into the CNN network as an
organic whole for more effective end-to-end optimization. The reasons are two
folds. It utilizes CRF to guide the message passing between pixels in
high-level features to purify the feature representation of boundary pixels,
with the help of inner pixels belonging to the same object. More importantly,
it enables optimizing class weights from both scale and direction during
backpropagation. We make detailed theoretical analysis to prove it. Besides,
superpixel is integrated into E-CRF and served as an auxiliary to exploit the
local object prior for more reliable message passing. Finally, our proposed
method yields impressive results on ADE20K, Cityscapes, and Pascal Context
datasets.
