  The expansion of artificial intelligence (AI) in pathology tasks has
intensified the demand for doctors' annotations in AI development. However,
collecting high-quality annotations from doctors is costly and time-consuming,
creating a bottleneck in AI progress. This study investigates eye-tracking as a
cost-effective technology to collect doctors' behavioral data for AI training
with a focus on the pathology task of mitosis detection. One major challenge in
using eye-gaze data is the low signal-to-noise ratio, which hinders the
extraction of meaningful information. We tackled this by levering the
properties of inter-observer eye-gaze consistencies and creating eye-gaze
labels from consistent eye-fixations shared by a group of observers. Our study
involved 14 non-medical participants, from whom we collected eye-gaze data and
generated eye-gaze labels based on varying group sizes. We assessed the
efficacy of such eye-gaze labels by training Convolutional Neural Networks
(CNNs) and comparing their performance to those trained with ground truth
annotations and a heuristic-based baseline. Results indicated that CNNs trained
with our eye-gaze labels closely followed the performance of ground-truth-based
CNNs, and significantly outperformed the baseline. Although primarily focused
on mitosis, we envision that insights from this study can be generalized to
other medical imaging tasks.
