  We present VILENS (Visual Inertial Lidar Legged Navigation System), an
odometry system for legged robots based on factor graphs. The key novelty is
the tight fusion of four different sensor modalities to achieve reliable
operation when the individual sensors would otherwise produce degenerate
estimation. To minimize leg odometry drift, we extend the robot's state with a
linear velocity bias term which is estimated online. This bias is observable
because of the tight fusion of this preintegrated velocity factor with vision,
lidar, and IMU factors. Extensive experimental validation on different ANYmal
quadruped robots is presented, for a total duration of 2 h and 1.8 km traveled.
The experiments involved dynamic locomotion over loose rocks, slopes, and mud
which caused challenges like slippage and terrain deformation. Perceptual
challenges included dark and dusty underground caverns, and open and
feature-deprived areas. We show an average improvement of 62% translational and
51% rotational errors compared to a state-of-the-art loosely coupled approach.
To demonstrate its robustness, VILENS was also integrated with a perceptive
controller and a local path planner.
