  Estimating the probability of failure is an important step in the
certification of safety-critical systems. Efficient estimation methods are
often needed due to the challenges posed by high-dimensional input spaces,
risky test scenarios, and computationally expensive simulators. This work
frames the problem of black-box safety validation as a Bayesian optimization
problem and introduces a method that iteratively fits a probabilistic surrogate
model to efficiently predict failures. The algorithm is designed to search for
failures, compute the most-likely failure, and estimate the failure probability
over an operating domain using importance sampling. We introduce three
acquisition functions that aim to reduce uncertainty by covering the design
space, optimize the analytically derived failure boundaries, and sample the
predicted failure regions. Results show this Bayesian safety validation
approach provides a more accurate estimate of failure probability with orders
of magnitude fewer samples and performs well across various safety validation
metrics. We demonstrate this approach on three test problems, a stochastic
decision making system, and a neural network-based runway detection system.
This work is open sourced (https://github.com/sisl/BayesianSafetyValidation.jl)
and currently being used to supplement the FAA certification process of the
machine learning components for an autonomous cargo aircraft.
