  Visual localization is one of the primary capabilities for mobile robots.
Long-term visual localization in real time is particularly challenging, in
which the robot is required to efficiently localize itself using visual data
where appearance may change significantly over time. In this paper, we propose
a cloud-based visual localization system targeting at long-term localization in
real time. On the robot, we employ two estimators to achieve accurate and
real-time performance. One is a sliding-window based visual inertial odometry,
which integrates constraints from consecutive observations and self-motion
measurements, as well as the constraints induced by localization on the cloud.
This estimator builds a local visual submap as the virtual observation which is
then sent to the cloud as new localization constraints. The other one is a
delayed state Extended Kalman Filter to fuse the pose of the robot localized
from the cloud, the local odometry and the high-frequency inertial
measurements. On the cloud, we propose a longer sliding-window based
localization method to aggregate the virtual observations for larger field of
view, leading to more robust alignment between virtual observations and the
map. Under this architecture, the robot can achieve drift-free and real-time
localization using onboard resources even in a network with limited bandwidth,
high latency and existence of package loss, which enables the autonomous
navigation in real-world environment. We evaluate the effectiveness of our
system on a dataset with challenging seasonal and illuminative variations. We
further validate the robustness of the system under challenging network
conditions.
