  Re-localizing a camera from a single image in a previously mapped area is
vital for many computer vision applications in robotics and augmented/virtual
reality. In this work, we address the problem of estimating the 6 DoF camera
pose relative to a global frame from a single image. We propose to leverage a
novel network of relative spatial and temporal geometric constraints to guide
the training of a Deep Network for localization. We employ simultaneously
spatial and temporal relative pose constraints that are obtained not only from
adjacent camera frames but also from camera frames that are distant in the
spatio-temporal space of the scene. We show that our method, through these
constraints, is capable of learning to localize when little or very sparse
ground-truth 3D coordinates are available. In our experiments, this is less
than 1% of available ground-truth data. We evaluate our method on 3 common
visual localization datasets and show that it outperforms other direct pose
estimation methods.
