  Autonomous driving has become real; semi-autonomous driving vehicles in an
affordable price range are already on the streets, and major automotive vendors
are actively developing full self-driving systems to deploy them in this
decade. Before rolling the products out to the end-users, it is critical to
test and ensure the safety of the autonomous driving systems, consisting of
multiple layers intertwined in a complicated way. However, while
safety-critical bugs may exist in any layer and even across layers, relatively
little attention has been given to testing the entire driving system across all
the layers. Prior work mainly focuses on white-box testing of individual layers
and preventing attacks on each layer.
  In this paper, we aim at holistic testing of autonomous driving systems that
have a whole stack of layers integrated in their entirety. Instead of looking
into the individual layers, we focus on the vehicle states that the system
continuously changes in the driving environment. This allows us to design
DriveFuzz, a new systematic fuzzing framework that can uncover potential
vulnerabilities regardless of their locations. DriveFuzz automatically
generates and mutates driving scenarios based on diverse factors leveraging a
high-fidelity driving simulator. We build novel driving test oracles based on
the real-world traffic rules to detect safety-critical misbehaviors, and guide
the fuzzer towards such misbehaviors through driving quality metrics referring
to the physical states of the vehicle.
  DriveFuzz has discovered 30 new bugs in various layers of two autonomous
driving systems (Autoware and CARLA Behavior Agent) and three additional bugs
in the CARLA simulator. We further analyze the impact of these bugs and how an
adversary may exploit them as security vulnerabilities to cause critical
accidents in the real world.
