  We conducted a lab-based eye-tracking study to investigate how the
interactivity of an AI-powered fact-checking system affects user interactions,
such as dwell time, attention, and mental resources involved in using the
system. A within-subject experiment was conducted, where participants used an
interactive and a non-interactive version of a mock AI fact-checking system and
rated their perceived correctness of COVID-19 related claims. We collected
web-page interactions, eye-tracking data, and mental workload using NASA-TLX.
We found that the presence of the affordance of interactively manipulating the
AI system's prediction parameters affected users' dwell times, and
eye-fixations on AOIs, but not mental workload. In the interactive system,
participants spent the most time evaluating claims' correctness, followed by
reading news. This promising result shows a positive role of interactivity in a
mixed-initiative AI-powered system.
