  In recent years, we have witnessed increasingly high performance in the field
of autonomous end-to-end driving. In particular, more and more research is
being done on driving in urban environments, where the car has to follow high
level commands to navigate. However, few evaluations are made on the ability of
these agents to react in an unexpected situation. Specifically, no evaluations
are conducted on the robustness of driving agents in the event of a bad
high-level command. We propose here an evaluation method, namely a benchmark
that allows to assess the robustness of an agent, and to appreciate its
understanding of the environment through its ability to keep a safe behavior,
regardless of the instruction.
