  Recently, sparse 3D convolutions have changed 3D object detection. Performing
on par with the voting-based approaches, 3D CNNs are memory-efficient and scale
to large scenes better. However, there is still room for improvement. With a
conscious, practice-oriented approach to problem-solving, we analyze the
performance of such methods and localize the weaknesses. Applying modifications
that resolve the found issues one by one, we end up with TR3D: a fast
fully-convolutional 3D object detection model trained end-to-end, that achieves
state-of-the-art results on the standard benchmarks, ScanNet v2, SUN RGB-D, and
S3DIS. Moreover, to take advantage of both point cloud and RGB inputs, we
introduce an early fusion of 2D and 3D features. We employ our fusion module to
make conventional 3D object detection methods multimodal and demonstrate an
impressive boost in performance. Our model with early feature fusion, which we
refer to as TR3D+FF, outperforms existing 3D object detection approaches on the
SUN RGB-D dataset. Overall, besides being accurate, both TR3D and TR3D+FF
models are lightweight, memory-efficient, and fast, thereby marking another
milestone on the way toward real-time 3D object detection. Code is available at
https://github.com/SamsungLabs/tr3d .
