  Simulation-based testing of automated driving systems (ADS) is the industry
standard, being a controlled, safe, and cost-effective alternative to
real-world testing. Despite these advantages, virtual simulations often fail to
accurately replicate real-world conditions like image fidelity, texture
representation, and environmental accuracy. This can lead to significant
differences in ADS behavior between simulated and real-world domains, a
phenomenon known as the sim2real gap. Researchers have used Image-to-Image
(I2I) neural translation to mitigate the sim2real gap, enhancing the realism of
simulated environments by transforming synthetic data into more authentic
representations of real-world conditions. However, while promising, these
techniques may potentially introduce artifacts, distortions, or inconsistencies
in the generated data that can affect the effectiveness of ADS testing. In our
empirical study, we investigated how the quality of image-to-image (I2I)
techniques influences the mitigation of the sim2real gap, using a set of
established metrics from the literature. We evaluated two popular generative
I2I architectures, pix2pix, and CycleGAN, across two ADS perception tasks at a
model level, namely vehicle detection and end-to-end lane keeping, using paired
simulated and real-world datasets. Our findings reveal that the effectiveness
of I2I architectures varies across different ADS tasks, and existing evaluation
metrics do not consistently align with the ADS behavior. Thus, we conducted
task-specific fine-tuning of perception metrics, which yielded a stronger
correlation. Our findings indicate that a perception metric that incorporates
semantic elements, tailored to each task, can facilitate selecting the most
appropriate I2I technique for a reliable assessment of the sim2real gap
mitigation.
