  Objective. Covert aspects of ongoing user mental states provide key context
information for user-aware human computer interactions. In this paper, we focus
on the problem of estimating the vigilance of users using EEG and EOG signals.
Approach. To improve the feasibility and wearability of vigilance estimation
devices for real-world applications, we adopt a novel electrode placement for
forehead EOG and extract various eye movement features, which contain the
principal information of traditional EOG. We explore the effects of EEG from
different brain areas and combine EEG and forehead EOG to leverage their
complementary characteristics for vigilance estimation. Considering that the
vigilance of users is a dynamic changing process because the intrinsic mental
states of users involve temporal evolution, we introduce continuous conditional
neural field and continuous conditional random field models to capture dynamic
temporal dependency. Main results. We propose a multimodal approach to
estimating vigilance by combining EEG and forehead EOG and incorporating the
temporal dependency of vigilance into model training. The experimental results
demonstrate that modality fusion can improve the performance compared with a
single modality, EOG and EEG contain complementary information for vigilance
estimation, and the temporal dependency-based models can enhance the
performance of vigilance estimation. From the experimental results, we observe
that theta and alpha frequency activities are increased, while gamma frequency
activities are decreased in drowsy states in contrast to awake states.
Significance. The forehead setup allows for the simultaneous collection of EEG
and EOG and achieves comparative performance using only four shared electrodes
in comparison with the temporal and posterior sites.
