  This paper proposes a robust localization system that employs deep learning
for better scene representation, and enhances the accuracy of 6-DOF camera pose
estimation. Inspired by the fact that global scene structure can be revealed by
wide field-of-view, we leverage the large overlap of a fisheye camera between
adjacent frames, and the powerful high-level feature representations of deep
learning. Our main contribution is the novel network architecture that extracts
both temporal and spatial information using a Recurrent Neural Network.
Specifically, we propose a novel pose regularization term combined with LSTM.
This leads to smoother pose estimation, especially for large outdoor scenery.
Promising experimental results on three benchmark datasets manifest the
effectiveness of the proposed approach.
