  Visual Inertial Odometry (VIO) is the task of estimating the movement
trajectory of an agent from an onboard camera stream fused with additional
Inertial Measurement Unit (IMU) measurements. A crucial subtask within VIO is
the tracking of features, which can be achieved through Optical Flow (OF). As
the calculation of OF is a resource-demanding task in terms of computational
load and memory footprint, which needs to be executed at low latency,
especially in robotic applications, OF estimation is today performed on
powerful CPUs or GPUs. This restricts its use in a broad spectrum of
applications where the deployment of such powerful, power-hungry processors is
unfeasible due to constraints related to cost, size, and power consumption.
On-sensor hardware acceleration is a promising approach to enable low latency
VIO even on resource-constrained devices such as nano drones. This paper
assesses the speed-up in a VIO sensor system exploiting a compact OF sensor
consisting of a global shutter camera and an Application Specific Integrated
Circuit (ASIC). By replacing the feature tracking logic of the VINS-Mono
pipeline with data from this OF camera, we demonstrate a 49.4% reduction in
latency and a 53.7% reduction of compute load of the VIO pipeline over the
original VINS-Mono implementation, allowing VINS-Mono operation up to 50 FPS
instead of 20 FPS on the quad-core ARM Cortex-A72 processor of a Raspberry Pi
Compute Module 4.
