  In this paper, we study in-depth the problem of online self-calibration for
robust and accurate visual-inertial state estimation. In particular, we first
perform a complete observability analysis for visual-inertial navigation
systems (VINS) with full calibration of sensing parameters, including IMU and
camera intrinsics and IMU-camera spatial-temporal extrinsic calibration, along
with readout time of rolling shutter (RS) cameras (if used). We investigate
different inertial model variants containing IMU intrinsic parameters that
encompass most commonly used models for low-cost inertial sensors. The
observability analysis results prove that VINS with full sensor calibration has
four unobservable directions, corresponding to the system's global yaw and
translation, while all sensor calibration parameters are observable given
fully-excited 6-axis motion. Moreover, we, for the first time, identify
primitive degenerate motions for IMU and camera intrinsic calibration. Each
degenerate motion profile will cause a set of calibration parameters to be
unobservable and any combination of these degenerate motions are still
degenerate. Extensive Monte-Carlo simulations and real-world experiments are
performed to validate both the observability analysis and identified degenerate
motions, showing that online self-calibration improves system accuracy and
robustness to calibration inaccuracies. We compare the proposed online
self-calibration on commonly-used IMUs against the state-of-art offline
calibration toolbox Kalibr, and show that the proposed system achieves better
consistency and repeatability. Based on our analysis and experimental
evaluations, we also provide practical guidelines for how to perform online
IMU-camera sensor self-calibration.
