  Driving in a state of drowsiness is a major cause of road accidents,
resulting in tremendous damage to life and property. Developing robust,
automatic, real-time systems that can infer drowsiness states of drivers has
the potential of making life-saving impact. However, developing drowsiness
detection systems that work well in real-world scenarios is challenging because
of the difficulties associated with collecting high-volume realistic drowsy
data and modeling the complex temporal dynamics of evolving drowsy states. In
this paper, we propose a data collection protocol that involves outfitting
vehicles of overnight shift workers with camera kits that record their faces
while driving. We develop a drowsiness annotation guideline to enable humans to
label the collected videos into 4 levels of drowsiness: `alert', `slightly
drowsy', `moderately drowsy' and `extremely drowsy'. We experiment with
different convolutional and temporal neural network architectures to predict
drowsiness states from pose, expression and emotion-based representation of the
input video of the driver's face. Our best performing model achieves a macro
ROC-AUC of 0.78, compared to 0.72 for a baseline model.
