  We present LM-Reloc -- a novel approach for visual relocalization based on
direct image alignment. In contrast to prior works that tackle the problem with
a feature-based formulation, the proposed method does not rely on feature
matching and RANSAC. Hence, the method can utilize not only corners but any
region of the image with gradients. In particular, we propose a loss
formulation inspired by the classical Levenberg-Marquardt algorithm to train
LM-Net. The learned features significantly improve the robustness of direct
image alignment, especially for relocalization across different conditions. To
further improve the robustness of LM-Net against large image baselines, we
propose a pose estimation network, CorrPoseNet, which regresses the relative
pose to bootstrap the direct image alignment. Evaluations on the CARLA and
Oxford RobotCar relocalization tracking benchmark show that our approach
delivers more accurate results than previous state-of-the-art methods while
being comparable in terms of robustness.
