  Data-driven simulators promise high data-efficiency for driving policy
learning. When used for modelling interactions, this data-efficiency becomes a
bottleneck: Small underlying datasets often lack interesting and challenging
edge cases for learning interactive driving. We address this challenge by
proposing a simulation method that uses in-painted ado vehicles for learning
robust driving policies. Thus, our approach can be used to learn policies that
involve multi-agent interactions and allows for training via state-of-the-art
policy learning methods. We evaluate the approach for learning standard
interaction scenarios in driving. In extensive experiments, our work
demonstrates that the resulting policies can be directly transferred to a
full-scale autonomous vehicle without making use of any traditional sim-to-real
transfer techniques such as domain randomization.
