  Our paper proposes a direct sparse visual odometry method that combines event
and RGB-D data to estimate the pose of agile-legged robots during dynamic
locomotion and acrobatic behaviors. Event cameras offer high temporal
resolution and dynamic range, which can eliminate the issue of blurred RGB
images during fast movements. This unique strength holds a potential for
accurate pose estimation of agile-legged robots, which has been a challenging
problem to tackle. Our framework leverages the benefits of both RGB-D and event
cameras to achieve robust and accurate pose estimation, even during dynamic
maneuvers such as jumping and landing a quadruped robot, the Mini-Cheetah. Our
major contributions are threefold: Firstly, we introduce an adaptive time
surface (ATS) method that addresses the whiteout and blackout issue in
conventional time surfaces by formulating pixel-wise decay rates based on scene
complexity and motion speed. Secondly, we develop an effective pixel selection
method that directly samples from event data and applies sample filtering
through ATS, enabling us to pick pixels on distinct features. Lastly, we
propose a nonlinear pose optimization formula that simultaneously performs
3D-2D alignment on both RGB-based and event-based maps and images, allowing the
algorithm to fully exploit the benefits of both data streams. We extensively
evaluate the performance of our framework on both public datasets and our own
quadruped robot dataset, demonstrating its effectiveness in accurately
estimating the pose of agile robots during dynamic movements.
