  Learning Bird's Eye View (BEV) representation from surrounding-view cameras
is of great importance for autonomous driving. In this work, we propose a
Geometry-guided Kernel Transformer (GKT), a novel 2D-to-BEV representation
learning mechanism. GKT leverages the geometric priors to guide the transformer
to focus on discriminative regions and unfolds kernel features to generate BEV
representation. For fast inference, we further introduce a look-up table (LUT)
indexing method to get rid of the camera's calibrated parameters at runtime.
GKT can run at $72.3$ FPS on 3090 GPU / $45.6$ FPS on 2080ti GPU and is robust
to the camera deviation and the predefined BEV height. And GKT achieves the
state-of-the-art real-time segmentation results, i.e., 38.0 mIoU
(100m$\times$100m perception range at a 0.5m resolution) on the nuScenes val
set. Given the efficiency, effectiveness, and robustness, GKT has great
practical values in autopilot scenarios, especially for real-time running
systems. Code and models will be available at
\url{https://github.com/hustvl/GKT}.
