  Systematic testing of autonomous vehicles operating in complex real-world
scenarios is a difficult and expensive problem. We present Paracosm, a reactive
language for writing test scenarios for autonomous driving systems. Paracosm
allows users to programmatically describe complex driving situations with
specific visual features, e.g., road layout in an urban environment, as well as
reactive temporal behaviors of cars and pedestrians. Paracosm programs are
executed on top of a game engine that provides realistic physics simulation and
visual rendering. The infrastructure allows systematic exploration of the state
space, both for visual features (lighting, shadows, fog) and for reactive
interactions with the environment (pedestrians, other traffic). We define a
notion of test coverage for Paracosm configurations based on combinatorial
testing and low dispersion sequences. Paracosm comes with an automatic test
case generator that uses random sampling for discrete parameters and
deterministic quasi-Monte Carlo generation for continuous parameters. Through
an empirical evaluation, we demonstrate the modeling and testing capabilities
of Paracosm on a suite of autonomous driving systems implemented using deep
neural networks developed in research and education. We show how Paracosm can
expose incorrect behaviors or degraded performance.
