  Information access systems are getting complex, and our understanding of user
behavior during information seeking processes is mainly drawn from qualitative
methods, such as observational studies or surveys. Leveraging the advances in
sensing technologies, our study aims to characterize user behaviors with
physiological signals, particularly in relation to cognitive load, affective
arousal, and valence. We conduct a controlled lab study with 26 participants,
and collect data including Electrodermal Activities, Photoplethysmogram,
Electroencephalogram, and Pupillary Responses. This study examines
informational search with four stages: the realization of Information Need
(IN), Query Formulation (QF), Query Submission (QS), and Relevance Judgment
(RJ). We also include different interaction modalities to represent modern
systems, e.g., QS by text-typing or verbalizing, and RJ with text or audio
information. We analyze the physiological signals across these stages and
report outcomes of pairwise non-parametric repeated-measure statistical tests.
The results show that participants experience significantly higher cognitive
loads at IN with a subtle increase in alertness, while QF requires higher
attention. QS involves demanding cognitive loads than QF. Affective responses
are more pronounced at RJ than QS or IN, suggesting greater interest and
engagement as knowledge gaps are resolved. To the best of our knowledge, this
is the first study that explores user behaviors in a search process employing a
more nuanced quantitative analysis of physiological signals. Our findings offer
valuable insights into user behavior and emotional responses in information
seeking processes. We believe our proposed methodology can inform the
characterization of more complex processes, such as conversational information
seeking.
