  Continuous estimation the driver's take-over readiness is critical for safe
and timely transfer of control during the failure modes of autonomous vehicles.
In this paper, we propose a data-driven approach for estimating the driver's
take-over readiness based purely on observable cues from in-vehicle vision
sensors. We present an extensive naturalistic drive dataset of drivers in a
conditionally autonomous vehicle running on Californian freeways. We collect
subjective ratings for the driver's take-over readiness from multiple human
observers viewing the sensor feed. Analysis of the ratings in terms of
intra-class correlation coefficients (ICCs) shows a high degree of consistency
in the ratings across raters. We define a metric for the driver's take-over
readiness termed the 'Observable Readiness Index (ORI)' based on the ratings.
Finally, we propose an LSTM model for continuous estimation of the driver's ORI
based on a holistic representation of the driver's state, capturing gaze, hand,
pose and foot activity. Our model estimates the ORI with a mean absolute error
of 0.449 on a 5 point scale.
