  For real-world driver drowsiness detection from videos, the variation of head
pose is so large that the existing methods on global face is not capable of
extracting effective features, such as looking aside and lowering head.
Temporal dependencies with variable length are also rarely considered by the
previous approaches, e.g., yawning and speaking. In this paper, we propose a
Long-term Multi-granularity Deep Framework to detect driver drowsiness in
driving videos containing the frontal faces. The framework includes two key
components: (1) Multi-granularity Convolutional Neural Network (MCNN), a novel
network utilizes a group of parallel CNN extractors on well-aligned facial
patches of different granularities, and extracts facial representations
effectively for large variation of head pose, furthermore, it can flexibly fuse
both detailed appearance clues of the main parts and local to global spatial
constraints; (2) a deep Long Short Term Memory network is applied on facial
representations to explore long-term relationships with variable length over
sequential frames, which is capable to distinguish the states with temporal
dependencies, such as blinking and closing eyes. Our approach achieves 90.05%
accuracy and about 37 fps speed on the evaluation set of the public NTHU-DDD
dataset, which is the state-of-the-art method on driver drowsiness detection.
Moreover, we build a new dataset named FI-DDD, which is of higher precision of
drowsy locations in temporal dimension.
