  Deep learning for clinical applications is subject to stringent performance
requirements, which raises a need for large labeled datasets. However, the
enormous cost of labeling medical data makes this challenging. In this paper,
we build a cost-sensitive active learning system for the problem of
intracranial hemorrhage detection and segmentation on head computed tomography
(CT). We show that our ensemble method compares favorably with the
state-of-the-art, while running faster and using less memory. Moreover, our
experiments are done using a substantially larger dataset than earlier papers
on this topic. Since the labeling time could vary tremendously across examples,
we model the labeling time and optimize the return on investment. We validate
this idea by core-set selection on our large labeled dataset and by growing it
with data from the wild.
