  Recent advancements in bird's eye view (BEV) representations have shown
remarkable promise for in-vehicle 3D perception. However, while these methods
have achieved impressive results on standard benchmarks, their robustness in
varied conditions remains insufficiently assessed. In this study, we present
RoboBEV, an extensive benchmark suite designed to evaluate the resilience of
BEV algorithms. This suite incorporates a diverse set of camera corruption
types, each examined over three severity levels. Our benchmarks also consider
the impact of complete sensor failures that occur when using multi-modal
models. Through RoboBEV, we assess 33 state-of-the-art BEV-based perception
models spanning tasks like detection, map segmentation, depth estimation, and
occupancy prediction. Our analyses reveal a noticeable correlation between the
model's performance on in-distribution datasets and its resilience to
out-of-distribution challenges. Our experimental results also underline the
efficacy of strategies like pre-training and depth-free BEV transformations in
enhancing robustness against out-of-distribution data. Furthermore, we observe
that leveraging extensive temporal information significantly improves the
model's robustness. Based on our observations, we design an effective
robustness enhancement strategy based on the CLIP model. The insights from this
study pave the way for the development of future BEV models that seamlessly
combine accuracy with real-world robustness.
