  Contemporary deep-learning object detection methods for autonomous driving
usually assume prefixed categories of common traffic participants, such as
pedestrians and cars. Most existing detectors are unable to detect uncommon
objects and corner cases (e.g., a dog crossing a street), which may lead to
severe accidents in some situations, making the timeline for the real-world
application of reliable autonomous driving uncertain. One main reason that
impedes the development of truly reliably self-driving systems is the lack of
public datasets for evaluating the performance of object detectors on corner
cases. Hence, we introduce a challenging dataset named CODA that exposes this
critical problem of vision-based detectors. The dataset consists of 1500
carefully selected real-world driving scenes, each containing four object-level
corner cases (on average), spanning more than 30 object categories. On CODA,
the performance of standard object detectors trained on large-scale autonomous
driving datasets significantly drops to no more than 12.8% in mAR. Moreover, we
experiment with the state-of-the-art open-world object detector and find that
it also fails to reliably identify the novel objects in CODA, suggesting that a
robust perception system for autonomous driving is probably still far from
reach. We expect our CODA dataset to facilitate further research in reliable
detection for real-world autonomous driving. Our dataset will be released at
https://coda-dataset.github.io.
