  For the foreseeble future, human beings will likely remain an integral part
of the driving task, monitoring the AI system as it performs anywhere from just
over 0% to just under 100% of the driving. The governing objectives of the MIT
Autonomous Vehicle Technology (MIT-AVT) study are to (1) undertake large-scale
real-world driving data collection that includes high-definition video to fuel
the development of deep learning based internal and external perception
systems, (2) gain a holistic understanding of how human beings interact with
vehicle automation technology by integrating video data with vehicle state
data, driver characteristics, mental models, and self-reported experiences with
technology, and (3) identify how technology and other factors related to
automation adoption and use can be improved in ways that save lives. In
pursuing these objectives, we have instrumented 23 Tesla Model S and Model X
vehicles, 2 Volvo S90 vehicles, 2 Range Rover Evoque, and 2 Cadillac CT6
vehicles for both long-term (over a year per driver) and medium term (one month
per driver) naturalistic driving data collection. Furthermore, we are
continually developing new methods for analysis of the massive-scale dataset
collected from the instrumented vehicle fleet. The recorded data streams
include IMU, GPS, CAN messages, and high-definition video streams of the driver
face, the driver cabin, the forward roadway, and the instrument cluster (on
select vehicles). The study is on-going and growing. To date, we have 122
participants, 15,610 days of participation, 511,638 miles, and 7.1 billion
video frames. This paper presents the design of the study, the data collection
hardware, the processing of the data, and the computer vision algorithms
currently being used to extract actionable knowledge from the data.
