  We focus on the task of far-field 3D detection (Far3Det) of objects beyond a
certain distance from an observer, e.g., $>$50m. Far3Det is particularly
important for autonomous vehicles (AVs) operating at highway speeds, which
require detections of far-field obstacles to ensure sufficient braking
distances. However, contemporary AV benchmarks such as nuScenes underemphasize
this problem because they evaluate performance only up to a certain distance
(50m). One reason is that obtaining far-field 3D annotations is difficult,
particularly for lidar sensors that produce very few point returns for far-away
objects. Indeed, we find that almost 50% of far-field objects (beyond 50m)
contain zero lidar points. Secondly, current metrics for 3D detection employ a
"one-size-fits-all" philosophy, using the same tolerance thresholds for near
and far objects, inconsistent with tolerances for both human vision and stereo
disparities. Both factors lead to an incomplete analysis of the Far3Det task.
For example, while conventional wisdom tells us that high-resolution RGB
sensors should be vital for 3D detection of far-away objects, lidar-based
methods still rank higher compared to RGB counterparts on the current benchmark
leaderboards. As a first step towards a Far3Det benchmark, we develop a method
to find well-annotated scenes from the nuScenes dataset and derive a
well-annotated far-field validation set. We also propose a Far3Det evaluation
protocol and explore various 3D detection methods for Far3Det. Our result
convincingly justifies the long-held conventional wisdom that high-resolution
RGB improves 3D detection in the far-field. We further propose a simple yet
effective method that fuses detections from RGB and lidar detectors based on
non-maximum suppression, which remarkably outperforms state-of-the-art 3D
detectors in the far-field.
