  Modern autonomous driving systems are typically divided into three main
tasks: perception, prediction, and planning. The planning task involves
predicting the trajectory of the ego vehicle based on inputs from both internal
intention and the external environment, and manipulating the vehicle
accordingly. Most existing works evaluate their performance on the nuScenes
dataset using the L2 error and collision rate between the predicted
trajectories and the ground truth. In this paper, we reevaluate these existing
evaluation metrics and explore whether they accurately measure the superiority
of different methods. Specifically, we design an MLP-based method that takes
raw sensor data (e.g., past trajectory, velocity, etc.) as input and directly
outputs the future trajectory of the ego vehicle, without using any perception
or prediction information such as camera images or LiDAR. Our simple method
achieves similar end-to-end planning performance on the nuScenes dataset with
other perception-based methods, reducing the average L2 error by about 20%.
Meanwhile, the perception-based methods have an advantage in terms of collision
rate. We further conduct in-depth analysis and provide new insights into the
factors that are critical for the success of the planning task on nuScenes
dataset. Our observation also indicates that we need to rethink the current
open-loop evaluation scheme of end-to-end autonomous driving in nuScenes. Codes
are available at https://github.com/E2E-AD/AD-MLP.
