  With the growing capabilities of autonomous vehicles, there is a higher
demand for sophisticated and pragmatic quality assurance approaches for machine
learning-enabled systems in the automotive AI context. The use of
simulation-based prototyping platforms provides the possibility for early-stage
testing, enabling inexpensive testing and the ability to capture critical
corner-case test scenarios. Simulation-based testing properly complements
conventional on-road testing. However, due to the large space of test input
parameters in these systems, the efficient generation of effective test
scenarios leading to the unveiling of failures is a challenge. This paper
presents a study on testing pedestrian detection and emergency braking system
of the Baidu Apollo autonomous driving platform within the SVL simulator. We
propose an evolutionary automated test generation technique that generates
failure-revealing scenarios for Apollo in the SVL environment. Our approach
models the input space using a generic and flexible data structure and benefits
a multi-criteria safety-based heuristic for the objective function targeted for
optimization. This paper presents the results of our proposed test generation
technique in the 2021 IEEE Autonomous Driving AI Test Challenge. In order to
demonstrate the efficiency and effectiveness of our approach, we also report
the results from a baseline random generation technique. Our evaluation shows
that the proposed evolutionary test case generator is more effective at
generating failure-revealing test cases and provides higher diversity between
the generated failures than the random baseline.
