  3D object detection in point clouds is a challenging vision task that
benefits various applications for understanding the 3D visual world. Lots of
recent research focuses on how to exploit end-to-end trainable Hough voting for
generating object proposals. However, the current voting strategy can only
receive partial votes from the surfaces of potential objects together with
severe outlier votes from the cluttered backgrounds, which hampers full
utilization of the information from the input point clouds. Inspired by the
back-tracing strategy in the conventional Hough voting methods, in this work,
we introduce a new 3D object detection method, named as Back-tracing
Representative Points Network (BRNet), which generatively back-traces the
representative points from the vote centers and also revisits complementary
seed points around these generated points, so as to better capture the fine
local structural features surrounding the potential objects from the raw point
clouds. Therefore, this bottom-up and then top-down strategy in our BRNet
enforces mutual consistency between the predicted vote centers and the raw
surface points and thus achieves more reliable and flexible object localization
and class prediction results. Our BRNet is simple but effective, which
significantly outperforms the state-of-the-art methods on two large-scale point
cloud datasets, ScanNet V2 (+7.5% in terms of<EMAIL_ADDRESS>and SUN RGB-D (+4.7% in
terms of mAP@0.50), while it is still lightweight and efficient. Code will be
available at https://github.com/cheng052/BRNet.
