  To be effective, state of the art machine learning technology needs large
amounts of annotated data. There are numerous compelling applications in
healthcare that can benefit from high performance automated decision support
systems provided by deep learning technology, but they lack the comprehensive
data resources required to apply sophisticated machine learning models.
Further, for economic reasons, it is very difficult to justify the creation of
large annotated corpora for these applications. Hence, automated annotation
techniques become increasingly important. In this study, we investigated the
effectiveness of using an active learning algorithm to automatically annotate a
large EEG corpus. The algorithm is designed to annotate six types of EEG
events. Two model training schemes, namely threshold-based and volume-based,
are evaluated. In the threshold-based scheme the threshold of confidence scores
is optimized in the initial training iteration, whereas for the volume-based
scheme only a certain amount of data is preserved after each iteration.
Recognition performance is improved 2% absolute and the system is capable of
automatically annotating previously unlabeled data. Given that the
interpretation of clinical EEG data is an exceedingly difficult task, this
study provides some evidence that the proposed method is a viable alternative
to expensive manual annotation.
