  Autonomous navigation at high speeds in off-road environments necessitates
robots to comprehensively understand their surroundings using onboard sensing
only. The extreme conditions posed by the off-road setting can cause degraded
camera image quality due to poor lighting and motion blur, as well as limited
sparse geometric information available from LiDAR sensing when driving at high
speeds. In this work, we present RoadRunner, a novel framework capable of
predicting terrain traversability and an elevation map directly from camera and
LiDAR sensor inputs. RoadRunner enables reliable autonomous navigation, by
fusing sensory information, handling of uncertainty, and generation of
contextually informed predictions about the geometry and traversability of the
terrain while operating at low latency. In contrast to existing methods relying
on classifying handcrafted semantic classes and using heuristics to predict
traversability costs, our method is trained end-to-end in a self-supervised
fashion. The RoadRunner network architecture builds upon popular sensor fusion
network architectures from the autonomous driving domain, which embed LiDAR and
camera information into a common Bird's Eye View perspective. Training is
enabled by utilizing an existing traversability estimation stack to generate
training data in hindsight in a scalable manner from real-world off-road
driving datasets. Furthermore, RoadRunner improves the system latency by a
factor of roughly 4, from 500 ms to 140 ms, while improving the accuracy for
traversability costs and elevation map predictions. We demonstrate the
effectiveness of RoadRunner in enabling safe and reliable off-road navigation
at high speeds in multiple real-world driving scenarios through unstructured
desert environments.
