  While most recent autonomous driving system focuses on developing perception
methods on ego-vehicle sensors, people tend to overlook an alternative approach
to leverage intelligent roadside cameras to extend the perception ability
beyond the visual range. We discover that the state-of-the-art vision-centric
bird's eye view detection methods have inferior performances on roadside
cameras. This is because these methods mainly focus on recovering the depth
regarding the camera center, where the depth difference between the car and the
ground quickly shrinks while the distance increases. In this paper, we propose
a simple yet effective approach, dubbed BEVHeight, to address this issue. In
essence, instead of predicting the pixel-wise depth, we regress the height to
the ground to achieve a distance-agnostic formulation to ease the optimization
process of camera-only perception methods. On popular 3D detection benchmarks
of roadside cameras, our method surpasses all previous vision-centric methods
by a significant margin. The code is available at
{\url{https://github.com/ADLab-AutoDrive/BEVHeight}}.
