  Disparity estimation for binocular stereo images finds a wide range of
applications. Traditional algorithms may fail on featureless regions, which
could be handled by high-level clues such as semantic segments. In this paper,
we suggest that appropriate incorporation of semantic cues can greatly rectify
prediction in commonly-used disparity estimation frameworks. Our method
conducts semantic feature embedding and regularizes semantic cues as the loss
term to improve learning disparity. Our unified model SegStereo employs
semantic features from segmentation and introduces semantic softmax loss, which
helps improve the prediction accuracy of disparity maps. The semantic cues work
well in both unsupervised and supervised manners. SegStereo achieves
state-of-the-art results on KITTI Stereo benchmark and produces decent
prediction on both CityScapes and FlyingThings3D datasets.
