  Real-world autonomous driving systems must make safe decisions in the face of
rare and diverse traffic scenarios. Current state-of-the-art planners are
mostly evaluated on real-world datasets like nuScenes (open-loop) or nuPlan
(closed-loop). In particular, nuPlan seems to be an expressive evaluation
method since it is based on real-world data and closed-loop, yet it mostly
covers basic driving scenarios. This makes it difficult to judge a planner's
capabilities to generalize to rarely-seen situations. Therefore, we propose a
novel closed-loop benchmark interPlan containing several edge cases and
challenging driving scenarios. We assess existing state-of-the-art planners on
our benchmark and show that neither rule-based nor learning-based planners can
safely navigate the interPlan scenarios.
  A recently evolving direction is the usage of foundation models like large
language models (LLM) to handle generalization. We evaluate an LLM-only planner
and introduce a novel hybrid planner that combines an LLM-based behavior
planner with a rule-based motion planner that achieves state-of-the-art
performance on our benchmark.
