  Multi-task learning has emerged as a powerful paradigm to solve a range of
tasks simultaneously with good efficiency in both computation resources and
inference time. However, these algorithms are designed for different tasks
mostly not within the scope of autonomous driving, thus making it hard to
compare multi-task methods in autonomous driving. Aiming to enable the
comprehensive evaluation of present multi-task learning methods in autonomous
driving, we extensively investigate the performance of popular multi-task
methods on the large-scale driving dataset, which covers four common perception
tasks, i.e., object detection, semantic segmentation, drivable area
segmentation, and lane detection. We provide an in-depth analysis of current
multi-task learning methods under different common settings and find out that
the existing methods make progress but there is still a large performance gap
compared with single-task baselines. To alleviate this dilemma in autonomous
driving, we present an effective multi-task framework, VE-Prompt, which
introduces visual exemplars via task-specific prompting to guide the model
toward learning high-quality task-specific representations. Specifically, we
generate visual exemplars based on bounding boxes and color-based markers,
which provide accurate visual appearances of target categories and further
mitigate the performance gap. Furthermore, we bridge transformer-based encoders
and convolutional layers for efficient and accurate unified perception in
autonomous driving. Comprehensive experimental results on the diverse
self-driving dataset BDD100K show that the VE-Prompt improves the multi-task
baseline and further surpasses single-task models.
