  Although autonomous vehicles (AVs) are expected to revolutionize
transportation, robust perception across a wide range of driving contexts
remains a significant challenge. Techniques to fuse sensor data from camera,
radar, and lidar sensors have been proposed to improve AV perception. However,
existing methods are insufficiently robust in difficult driving contexts (e.g.,
bad weather, low light, sensor obstruction) due to rigidity in their fusion
implementations. These methods fall into two broad categories: (i) early
fusion, which fails when sensor data is noisy or obscured, and (ii) late
fusion, which cannot leverage features from multiple sensors and thus produces
worse estimates. To address these limitations, we propose HydraFusion: a
selective sensor fusion framework that learns to identify the current driving
context and fuses the best combination of sensors to maximize robustness
without compromising efficiency. HydraFusion is the first approach to propose
dynamically adjusting between early fusion, late fusion, and combinations
in-between, thus varying both how and when fusion is applied. We show that, on
average, HydraFusion outperforms early and late fusion approaches by 13.66% and
14.54%, respectively, without increasing computational complexity or energy
consumption on the industry-standard Nvidia Drive PX2 AV hardware platform. We
also propose and evaluate both static and deep-learning-based context
identification strategies. Our open-source code and model implementation are
available at https://github.com/AICPS/hydrafusion.
