  Estimating the 3D structure of the drivable surface and surrounding
environment is a crucial task for assisted and autonomous driving. It is
commonly solved either by using 3D sensors such as LiDAR or directly predicting
the depth of points via deep learning. However, the former is expensive, and
the latter lacks the use of geometry information for the scene. In this paper,
instead of following existing methodologies, we propose Road Planar Parallax
Attention Network (RPANet), a new deep neural network for 3D sensing from
monocular image sequences based on planar parallax, which takes full advantage
of the omnipresent road plane geometry in driving scenes. RPANet takes a pair
of images aligned by the homography of the road plane as input and outputs a
$\gamma$ map (the ratio of height to depth) for 3D reconstruction. The $\gamma$
map has the potential to construct a two-dimensional transformation between two
consecutive frames. It implies planar parallax and can be combined with the
road plane serving as a reference to estimate the 3D structure by warping the
consecutive frames. Furthermore, we introduce a novel cross-attention module to
make the network better perceive the displacements caused by planar parallax.
To verify the effectiveness of our method, we sample data from the Waymo Open
Dataset and construct annotations related to planar parallax. Comprehensive
experiments are conducted on the sampled dataset to demonstrate the 3D
reconstruction accuracy of our approach in challenging scenarios.
