  Developing and fielding complex systems requires proof that they are reliably
correct with respect to their design and operating requirements. Especially for
autonomous systems which exhibit unanticipated emergent behavior, fully
enumerating the range of possible correct and incorrect behaviors is
intractable. Therefore, we propose an optimization-based search technique for
generating high-quality, high-variance, and non-trivial data which captures the
range of correct and incorrect responses a system could exhibit. This manifold
between desired and undesired behavior provides a more detailed understanding
of system reliability than traditional testing or Monte Carlo simulations.
After discovering data points along the manifold, we apply machine learning
techniques to quantify the decision manifold's underlying mathematical
function. Such models serve as correctness properties which can be utilized to
enable both verification during development and testing, as well as continuous
assurance during operation, even amidst system adaptations and dynamic
operating environments. This method can be applied in combination with a
simulator in order to provide evidence of dependability to system designers and
users, with the ultimate aim of establishing trust in the deployment of complex
systems. In this proof-of-concept, we apply our method to a
software-in-the-loop evaluation of an autonomous vehicle.
