  Testing with simulation environments helps to identify critical failing
scenarios for self-driving cars (SDCs). Simulation-based tests are safer than
in-field operational tests and allow detecting software defects before
deployment. However, these tests are very expensive and are too many to be run
frequently within limited time constraints. In this paper, we investigate test
case prioritization techniques to increase the ability to detect SDC regression
faults with virtual tests earlier. Our approach, called SDC-Prioritizer,
prioritizes virtual tests for SDCs according to static features of the roads we
designed to be used within the driving scenarios. These features can be
collected without running the tests, which means that they do not require past
execution results. We introduce two evolutionary approaches to prioritize the
test cases using diversity metrics (black-box heuristics) computed on these
static features. These two approaches, called SO-SDC-Prioritizer and
MO-SDC-Prioritizer, use single-objective and multi-objective genetic
algorithms, respectively, to find trade-offs between executing the less
expensive tests and the most diverse test cases earlier. Our empirical study
conducted in the SDC domain shows that MO-SDC-Prioritizer significantly
improves the ability to detect safety-critical failures at the same level of
execution time compared to baselines: random and greedy-based test case
orderings. Besides, our study indicates that multi-objective meta-heuristics
outperform single-objective approaches when prioritizing simulation-based tests
for SDCs. MO-SDC-Prioritizer prioritizes test cases with a large improvement in
fault detection while its overhead (up to 0.45% of the test execution cost) is
negligible.
