  Neural networks have become state-of-the-art for computer vision problems
because of their ability to efficiently model complex functions from large
amounts of data. While neural networks can be shown to perform well empirically
for a variety of tasks, their performance is difficult to guarantee. Neural
network verification tools have been developed that can certify robustness with
respect to a given input image; however, for neural network systems used in
closed-loop controllers, robustness with respect to individual images does not
address multi-step properties of the neural network controller and its
environment. Furthermore, neural network systems interacting in the physical
world and using natural images are operating in a black-box environment, making
formal verification intractable. This work combines the adaptive stress testing
(AST) framework with neural network verification tools to search for the most
likely sequence of image disturbances that cause the neural network controlled
system to reach a failure. An autonomous aircraft taxi application is
presented, and results show that the AST method finds failures with more likely
image disturbances than baseline methods. Further analysis of AST results
revealed an explainable cause of the failure, giving insight into the
problematic scenarios that should be addressed.
