  Think aloud protocols are widely applied in user experience studies. In this
paper, the effect of two different applications of the Retrospective Think
Aloud (RTA) protocol on the number of user-reported usability issues is
examined. To this end, 30 users were asked to use the National Cadastre and
Mapping Agency web application and complete a set of tasks, such as measuring
the land area of a square in their hometown. The order of tasks was randomized
per participant. Next, participants were involved in RTA sessions. Each
participant was involved in two different RTA modes: (a) the strict guidance,
in which the facilitator stayed in the background and prompted participants to
keep thinking aloud based on his judgement and experience, and (b) the
physiology-supported interventions, in which the facilitator intervened based
on real-time monitoring of user's physiological signals. During each session,
three participant's physiological signals were recorded: skin conductance, skin
temperature and blood volume pulse. Participants were also asked to provide
valence-arousal ratings for each self-reported usability issue. Analysis of the
collected data showed that participants in the physiology-supported RTA mode
reported significantly more usability issues. No significant effect of the RTA
mode was found on the va-lence-arousal ratings for the reported usability
issues. Participants' physiological signals during the RTA sessions did not
also differ significantly between the two modes.
