  Parsing urban scene images benefits many applications, especially
self-driving. Most of the current solutions employ generic image parsing models
that treat all scales and locations in the images equally and do not consider
the geometry property of car-captured urban scene images. Thus, they suffer
from heterogeneous object scales caused by perspective projection of cameras on
actual scenes and inevitably encounter parsing failures on distant objects as
well as other boundary and recognition errors. In this work, we propose a new
FoveaNet model to fully exploit the perspective geometry of scene images and
address the common failures of generic parsing models. FoveaNet estimates the
perspective geometry of a scene image through a convolutional network which
integrates supportive evidence from contextual objects within the image. Based
on the perspective geometry information, FoveaNet "undoes" the camera
perspective projection analyzing regions in the space of the actual scene, and
thus provides much more reliable parsing results. Furthermore, to effectively
address the recognition errors, FoveaNet introduces a new dense CRFs model that
takes the perspective geometry as a prior potential. We evaluate FoveaNet on
two urban scene parsing datasets, Cityspaces and CamVid, which demonstrates
that FoveaNet can outperform all the well-established baselines and provide new
state-of-the-art performance.
