  A typical scenario-based evaluation framework seeks to characterize a
black-box system's safety performance (e.g., failure rate) through repeatedly
sampling initialization configurations (scenario sampling) and executing a
certain test policy for scenario propagation (scenario testing) with the
black-box system involved as the test subject. In this letter, we first present
a novel safety evaluation criterion that seeks to characterize the actual
operational domain within which the test subject would remain safe indefinitely
with high probability. By formulating the black-box testing scenario as a
dynamic system, we show that the presented problem is equivalent to finding a
certain "almost" robustly forward invariant set for the given system. Second,
for an arbitrary scenario testing strategy, we propose a scenario sampling
algorithm that is provably asymptotically optimal in obtaining the safe
invariant set with arbitrarily high accuracy. Moreover, as one considers
different testing strategies (e.g., biased sampling of safety-critical cases),
we show that the proposed algorithm still converges to the unbiased
approximation of the safety characterization outcome if the scenario testing
satisfies a certain condition. Finally, the effectiveness of the presented
scenario sampling algorithms and various theoretical properties are
demonstrated in a case study of the safety evaluation of a control barrier
function-based mobile robot collision avoidance system.
