  Large Vision-Language Models (LVLMs) have received widespread attention in
advancing the interpretable self-driving. Existing evaluations of LVLMs
primarily focus on the multi-faceted capabilities in natural circumstances,
lacking automated and quantifiable assessment for self-driving, let alone the
severe road corner cases. In this paper, we propose CODA-LM, the very first
benchmark for the automatic evaluation of LVLMs for self-driving corner cases.
We adopt a hierarchical data structure to prompt powerful LVLMs to analyze
complex driving scenes and generate high-quality pre-annotation for human
annotators, and for LVLM evaluation, we show that using the text-only large
language models (LLMs) as judges reveals even better alignment with human
preferences than the LVLM judges. Moreover, with CODA-LM, we build CODA-VLM, a
new driving LVLM surpassing all the open-sourced counterparts on CODA-LM. Our
CODA-VLM performs comparably with GPT-4V, even surpassing GPT-4V by +21.42% on
the regional perception task. We hope CODA-LM can become the catalyst to
promote interpretable self-driving empowered by LVLMs.
