  Multimodal data analysis and validation based on streams from
state-of-the-art sensor technology such as eye-tracking or emotion recognition
using the Facial Action Coding System (FACTs) with deep learning allows
educational researchers to study multifaceted learning and problem-solving
processes and to improve educational experiences. This study aims to
investigate the correlation between two continuous sensor streams, pupil
diameter as an indicator of cognitive workload and FACTs with deep learning as
an indicator of emotional arousal (RQ 1a), specifically for epochs of high,
medium, and low arousal (RQ 1b). Furthermore, the time lag between emotional
arousal and pupil diameter data will be analyzed (RQ 2). 28 participants worked
on three cognitively demanding and emotionally engaging everyday moral dilemmas
while eye-tracking and emotion recognition data were collected. The data were
pre-processed in Phyton (synchronization, blink control, downsampling) and
analyzed using correlation analysis and Granger causality tests. The results
show negative and statistically significant correlations between the data
streams for emotional arousal and pupil diameter. However, the correlation is
negative and significant only for epochs of high arousal, while positive but
non-significant relationships were found for epochs of medium or low arousal.
The average time lag for the relationship between arousal and pupil diameter
was 2.8 ms. In contrast to previous findings without a multimodal approach
suggesting a positive correlation between the constructs, the results
contribute to the state of research by highlighting the importance of
multimodal data validation and research on convergent vagility. Future research
should consider emotional regulation strategies and emotional valence.
