  Simulation-based falsification is a practical testing method to increase
confidence that the system will meet safety requirements. Because full-fidelity
simulations can be computationally demanding, we investigate the use of
simulators with different levels of fidelity. As a first step, we express the
overall safety specification in terms of environmental parameters and structure
this safety specification as an optimization problem. We propose a
multi-fidelity falsification framework using Bayesian optimization, which is
able to determine at which level of fidelity we should conduct a safety
evaluation in addition to finding possible instances from the environment that
cause the system to fail. This method allows us to automatically switch between
inexpensive, inaccurate information from a low-fidelity simulator and
expensive, accurate information from a high-fidelity simulator in a
cost-effective way. Our experiments on various environments in simulation
demonstrate that multi-fidelity Bayesian optimization has falsification
performance comparable to single-fidelity Bayesian optimization but with much
lower cost.
