  Detecting objects such as cars and pedestrians in 3D plays an indispensable
role in autonomous driving. Existing approaches largely rely on expensive LiDAR
sensors for accurate depth information. While recently pseudo-LiDAR has been
introduced as a promising alternative, at a much lower cost based solely on
stereo images, there is still a notable performance gap. In this paper we
provide substantial advances to the pseudo-LiDAR framework through improvements
in stereo depth estimation. Concretely, we adapt the stereo network
architecture and loss function to be more aligned with accurate depth
estimation of faraway objects --- currently the primary weakness of
pseudo-LiDAR. Further, we explore the idea to leverage cheaper but extremely
sparse LiDAR sensors, which alone provide insufficient information for 3D
detection, to de-bias our depth estimation. We propose a depth-propagation
algorithm, guided by the initial depth estimates, to diffuse these few exact
measurements across the entire depth map. We show on the KITTI object detection
benchmark that our combined approach yields substantial improvements in depth
estimation and stereo-based 3D object detection --- outperforming the previous
state-of-the-art detection accuracy for faraway objects by 40%. Our code is
available at https://github.com/mileyan/Pseudo_Lidar_V2.
