  3D object detection from a single image without LiDAR is a challenging task
due to the lack of accurate depth information. Conventional 2D convolutions are
unsuitable for this task because they fail to capture local object and its
scale information, which are vital for 3D object detection. To better represent
3D structure, prior arts typically transform depth maps estimated from 2D
images into a pseudo-LiDAR representation, and then apply existing 3D
point-cloud based object detectors. However, their results depend heavily on
the accuracy of the estimated depth maps, resulting in suboptimal performance.
In this work, instead of using pseudo-LiDAR representation, we improve the
fundamental 2D fully convolutions by proposing a new local convolutional
network (LCN), termed Depth-guided Dynamic-Depthwise-Dilated LCN (D$^4$LCN),
where the filters and their receptive fields can be automatically learned from
image-based depth maps, making different pixels of different images have
different filters. D$^4$LCN overcomes the limitation of conventional 2D
convolutions and narrows the gap between image representation and 3D point
cloud representation. Extensive experiments show that D$^4$LCN outperforms
existing works by large margins. For example, the relative improvement of
D$^4$LCN against the state-of-the-art on KITTI is 9.1\% in the moderate
setting. The code is available at https://github.com/dingmyu/D4LCN.
