  Localization is paramount for autonomous robots. While camera and LiDAR-based
approaches have been extensively investigated, they are affected by adverse
illumination and weather conditions. Therefore, radar sensors have recently
gained attention due to their intrinsic robustness to such conditions. In this
paper, we propose RaLF, a novel deep neural network-based approach for
localizing radar scans in a LiDAR map of the environment, by jointly learning
to address both place recognition and metric localization. RaLF is composed of
radar and LiDAR feature encoders, a place recognition head that generates
global descriptors, and a metric localization head that predicts the 3-DoF
transformation between the radar scan and the map. We tackle the place
recognition task by learning a shared embedding space between the two
modalities via cross-modal metric learning. Additionally, we perform metric
localization by predicting pixel-level flow vectors that align the query radar
scan with the LiDAR map. We extensively evaluate our approach on multiple
real-world driving datasets and show that RaLF achieves state-of-the-art
performance for both place recognition and metric localization. Moreover, we
demonstrate that our approach can effectively generalize to different cities
and sensor setups than the ones used during training. We make the code and
trained models publicly available at http://ralf.cs.uni-freiburg.de.
