  ComOpT is an open-source research tool for coverage-driven testing of
autonomous driving systems, focusing on planning and control. Starting with (i)
a meta-model characterizing discrete conditions to be considered and (ii)
constraints specifying the impossibility of certain combinations, ComOpT first
generates constraint-feasible abstract scenarios while maximally increasing the
coverage of k-way combinatorial testing. Each abstract scenario can be viewed
as a conceptual equivalence class, which is then instantiated into multiple
concrete scenarios by (1) randomly picking one local map that fulfills the
specified geographical condition, and (2) assigning all actors accordingly with
parameters within the range. Finally, ComOpT evaluates each concrete scenario
against a set of KPIs and performs local scenario variation via spawning a new
agent that might lead to a collision at designated points. We use ComOpT to
test the Apollo~6 autonomous driving software stack. ComOpT can generate highly
diversified scenarios with limited test budgets while uncovering problematic
situations such as inabilities to make simple right turns, uncomfortable
accelerations, and dangerous driving patterns. ComOpT participated in the 2021
IEEE AI Autonomous Vehicle Testing Challenge and won first place among more
than 110 contending teams.
