  3D multi-object detection and tracking are crucial for traffic scene
understanding. However, the community pays less attention to these areas due to
the lack of a standardized benchmark dataset to advance the field. Moreover,
existing datasets (e.g., KITTI) do not provide sufficient data and labels to
tackle challenging scenes where highly interactive and occluded traffic
participants are present. To address the issues, we present the Honda Research
Institute 3D Dataset (H3D), a large-scale full-surround 3D multi-object
detection and tracking dataset collected using a 3D LiDAR scanner. H3D
comprises of 160 crowded and highly interactive traffic scenes with a total of
1 million labeled instances in 27,721 frames. With unique dataset size, rich
annotations, and complex scenes, H3D is gathered to stimulate research on
full-surround 3D multi-object detection and tracking. To effectively and
efficiently annotate a large-scale 3D point cloud dataset, we propose a
labeling methodology to speed up the overall annotation cycle. A standardized
benchmark is created to evaluate full-surround 3D multi-object detection and
tracking algorithms. 3D object detection and tracking algorithms are trained
and tested on H3D. Finally, sources of errors are discussed for the development
of future algorithms.
