  Virtual testing is a crucial task to ensure safety in autonomous driving, and
sensor simulation is an important task in this domain. Most current LiDAR
simulations are very simplistic and are mainly used to perform initial tests,
while the majority of insights are gathered on the road. In this paper, we
propose a lightweight approach for more realistic LiDAR simulation that learns
a real sensor's behavior from test drive data and transforms this to the
virtual domain. The central idea is to cast the simulation into an
image-to-image translation problem. We train our pix2pix based architecture on
two real world data sets, namely the popular KITTI data set and the Audi
Autonomous Driving Dataset which provide both, RGB and LiDAR images. We apply
this network on synthetic renderings and show that it generalizes sufficiently
from real images to simulated images. This strategy enables to skip the
sensor-specific, expensive and complex LiDAR physics simulation in our
synthetic world and avoids oversimplification and a large domain-gap through
the clean synthetic environment.
