  Recognizing and localizing objects in the 3D space is a crucial ability for
an AI agent to perceive its surrounding environment. While significant progress
has been achieved with expensive LiDAR point clouds, it poses a great challenge
for 3D object detection given only a monocular image. While there exist
different alternatives for tackling this problem, it is found that they are
either equipped with heavy networks to fuse RGB and depth information or
empirically ineffective to process millions of pseudo-LiDAR points. With
in-depth examination, we realize that these limitations are rooted in
inaccurate object localization. In this paper, we propose a novel and
lightweight approach, dubbed {\em Progressive Coordinate Transforms} (PCT) to
facilitate learning coordinate representations. Specifically, a localization
boosting mechanism with confidence-aware loss is introduced to progressively
refine the localization prediction. In addition, semantic image representation
is also exploited to compensate for the usage of patch proposals. Despite being
lightweight and simple, our strategy leads to superior improvements on the
KITTI and Waymo Open Dataset monocular 3D detection benchmarks. At the same
time, our proposed PCT shows great generalization to most coordinate-based 3D
detection frameworks. The code is available at:
https://github.com/amazon-research/progressive-coordinate-transforms .
