  Uncovering potential failure cases is a crucial step in the validation of
safety critical systems such as autonomous vehicles. Failure search may be done
through logging substantial vehicle miles in either simulation or real world
testing. Due to the sparsity of failure events, naive random search approaches
require significant amounts of vehicle operation hours to find potential system
weaknesses. As a result, adaptive searching techniques have been proposed to
efficiently explore and uncover failure trajectories of an autonomous policy in
simulation. Adaptive Stress Testing (AST) is one such method that poses the
problem of failure search as a Markov decision process and uses reinforcement
learning techniques to find high probability failures. However, this
formulation requires a probability model for the actions of all agents in the
environment. In systems where the environment actions are discrete and
dependencies among agents exist, it may be infeasible to fully characterize the
distribution or find a suitable proxy. This work proposes the use of a data
driven approach to learn a suitable classifier that tries to model how humans
identify {critical states and use this to guide failure search in AST. We show
that the incorporation of critical states into the AST framework generates
failure scenarios with increased safety violations in an autonomous driving
policy with a discrete action space.
