  Autonomous systems such as self-driving cars and general-purpose robots are
safety-critical systems that operate in highly uncertain and dynamic
environments. We propose an interactive multi-agent framework where the
system-under-design is modeled as an ego agent and its environment is modeled
by a number of adversarial (ado) agents. For example, a self-driving car is an
ego agent whose behavior is influenced by ado agents such as pedestrians,
bicyclists, traffic lights, road geometry etc. Given a logical specification of
the correct behavior of the ego agent, and a set of constraints that encode
reasonable adversarial behavior, our framework reduces the adversarial testing
problem to the problem of synthesizing controllers for (constrained) ado agents
that cause the ego agent to violate its specifications. Specifically, we
explore the use of tabular and deep reinforcement learning approaches for
synthesizing adversarial agents. We show that ado agents trained in this
fashion are better than traditional falsification or testing techniques because
they can generalize to ego agents and environments that differ from the
original ego agent. We demonstrate the efficacy of our technique on two
real-world case studies from the domain of self-driving cars.
