  High-precision vehicle localization with commercial setups is a crucial
technique for high-level autonomous driving tasks. Localization with a
monocular camera in LiDAR map is a newly emerged approach that achieves
promising balance between cost and accuracy, but estimating pose by finding
correspondences between such cross-modal sensor data is challenging, thereby
damaging the localization accuracy. In this paper, we address the problem by
proposing a novel Transformer-based neural network to register 2D images into
3D LiDAR map in an end-to-end manner. Poses are implicitly represented as
high-dimensional feature vectors called pose queries and can be iteratively
updated by interacting with the retrieved relevant information from cross-model
features using attention mechanism in a proposed POse Estimator Transformer
(POET) module. Moreover, we apply a multiple hypotheses aggregation method that
estimates the final poses by performing parallel optimization on multiple
randomly initialized pose queries to reduce the network uncertainty.
Comprehensive analysis and experimental results on public benchmark conclude
that the proposed image-to-LiDAR map localization network could achieve
state-of-the-art performances in challenging cross-modal localization tasks.
