  Simulators are widely used to test Autonomous Driving Systems (ADS), but
their potential flakiness can lead to inconsistent test results. We investigate
test flakiness in simulation-based testing of ADS by addressing two key
questions: (1) How do flaky ADS simulations impact automated testing that
relies on randomized algorithms? and (2) Can machine learning (ML) effectively
identify flaky ADS tests while decreasing the required number of test reruns?
Our empirical results, obtained from two widely-used open-source ADS simulators
and five diverse ADS test setups, show that test flakiness in ADS is a common
occurrence and can significantly impact the test results obtained by randomized
algorithms. Further, our ML classifiers effectively identify flaky ADS tests
using only a single test run, achieving F1-scores of $85$%, $82$% and $96$% for
three different ADS test setups. Our classifiers significantly outperform our
non-ML baseline, which requires executing tests at least twice, by $31$%,
$21$%, and $13$% in F1-score performance, respectively. We conclude with a
discussion on the scope, implications and limitations of our study. We provide
our complete replication package in a Github repository.
