  Autonomous Vehicles (AVs) promise a range of societal advantages, including
broader access to mobility, reduced road accidents, and enhanced transportation
efficiency. However, evaluating the risks linked to AVs is complex due to
limited historical data and the swift progression of technology. This paper
presents a data-driven framework for assessing the risk of different AVs'
behaviors in various operational design domains (ODDs), based on counterfactual
simulations of "misbehaving" road users. We propose the notion of
counterfactual safety margin, which represents the minimum deviation from
nominal behavior that could cause a collision. This methodology not only
pinpoints the most critical scenarios but also quantifies the (relative) risk's
frequency and severity concerning AVs. Importantly, we show that our approach
is applicable even when the AV's behavioral policy remains undisclosed, through
worst- and best-case analyses, benefiting external entities like regulators and
risk evaluators. Our experimental outcomes demonstrate the correlation between
the safety margin, the quality of the driving policy, and the ODD, shedding
light on the relative risks of different AV providers. Overall, this work
contributes to the safety assessment of AVs and addresses legislative and
insurance concerns surrounding this burgeoning technology.
