  Applying curriculum learning requires both a range of difficulty in data and
a method for determining the difficulty of examples. In many tasks, however,
satisfying these requirements can be a formidable challenge. In this paper, we
contend that histopathology image classification is a compelling use case for
curriculum learning. Based on the nature of histopathology images, a range of
difficulty inherently exists among examples, and, since medical datasets are
often labeled by multiple annotators, annotator agreement can be used as a
natural proxy for the difficulty of a given example. Hence, we propose a simple
curriculum learning method that trains on progressively-harder images as
determined by annotator agreement. We evaluate our hypothesis on the
challenging and clinically-important task of colorectal polyp classification.
Whereas vanilla training achieves an AUC of 83.7% for this task, a model
trained with our proposed curriculum learning approach achieves an AUC of
88.2%, an improvement of 4.5%. Our work aims to inspire researchers to think
more creatively and rigorously when choosing contexts for applying curriculum
learning.
