  Driving Scene understanding is a key ingredient for intelligent
transportation systems. To achieve systems that can operate in a complex
physical and social environment, they need to understand and learn how humans
drive and interact with traffic scenes. We present the Honda Research Institute
Driving Dataset (HDD), a challenging dataset to enable research on learning
driver behavior in real-life environments. The dataset includes 104 hours of
real human driving in the San Francisco Bay Area collected using an
instrumented vehicle equipped with different sensors. We provide a detailed
analysis of HDD with a comparison to other driving datasets. A novel annotation
methodology is introduced to enable research on driver behavior understanding
from untrimmed data sequences. As the first step, baseline algorithms for
driver behavior detection are trained and tested to demonstrate the feasibility
of the proposed task.
