  When deep neural network (DNN) was first introduced to the medical image
analysis community, researchers were impressed by its performance. However, it
is evident now that a large number of manually labeled data is often a must to
train a properly functioning DNN. This demand for supervision data and labels
is a major bottleneck in current medical image analysis, since collecting a
large number of annotations from experienced experts can be time-consuming and
expensive. In this paper, we demonstrate that the eye movement of radiologists
reading medical images can be a new form of supervision to train the DNN-based
computer-aided diagnosis (CAD) system. Particularly, we record the tracks of
the radiologists' gaze when they are reading images. The gaze information is
processed and then used to supervise the DNN's attention via an Attention
Consistency module. To the best of our knowledge, the above pipeline is among
the earliest efforts to leverage expert eye movement for deep-learning-based
CAD. We have conducted extensive experiments on knee X-ray images for
osteoarthritis assessment. The results show that our method can achieve
considerable improvement in diagnosis performance, with the help of gaze
supervision.
