  In this work, we present a simple end-to-end trainable machine learning
system capable of realistically simulating driving experiences. This can be
used for the verification of self-driving system performance without relying on
expensive and time-consuming road testing. In particular, we frame the
simulation problem as a Markov Process, leveraging deep neural networks to
model both state distribution and transition function. These are trainable
directly from the existing raw observations without the need for any
handcrafting in the form of plant or kinematic models. All that is needed is a
dataset of historical traffic episodes. Our formulation allows the system to
construct never seen scenes that unfold realistically reacting to the
self-driving car's behaviour. We train our system directly from 1,000 hours of
driving logs and measure both realism, reactivity of the simulation as the two
key properties of the simulation. At the same time, we apply the method to
evaluate the performance of a recently proposed state-of-the-art ML planning
system trained from human driving logs. We discover this planning system is
prone to previously unreported causal confusion issues that are difficult to
test by non-reactive simulation. To the best of our knowledge, this is the
first work that directly merges highly realistic data-driven simulations with a
closed-loop evaluation for self-driving vehicles. We make the data, code, and
pre-trained models publicly available to further stimulate simulation
development.
