  Towards 3D object tracking in point clouds, a novel point-to-box network
termed P2B is proposed in an end-to-end learning manner. Our main idea is to
first localize potential target centers in 3D search area embedded with target
information. Then point-driven 3D target proposal and verification are executed
jointly. In this way, the time-consuming 3D exhaustive search can be avoided.
Specifically, we first sample seeds from the point clouds in template and
search area respectively. Then, we execute permutation-invariant feature
augmentation to embed target clues from template into search area seeds and
represent them with target-specific features. Consequently, the augmented
search area seeds regress the potential target centers via Hough voting. The
centers are further strengthened with seed-wise targetness scores. Finally,
each center clusters its neighbors to leverage the ensemble power for joint 3D
target proposal and verification. We apply PointNet++ as our backbone and
experiments on KITTI tracking dataset demonstrate P2B's superiority (~10%'s
improvement over state-of-the-art). Note that P2B can run with 40FPS on a
single NVIDIA 1080Ti GPU. Our code and model are available at
https://github.com/HaozheQi/P2B.
