  Efficient label acquisition processes are key to obtaining robust
classifiers. However, data labeling is often challenging and subject to high
levels of label noise. This can arise even when classification targets are well
defined, if instances to be labeled are more difficult than the prototypes used
to define the class, leading to disagreements among the expert community. Here,
we enable efficient training of deep neural networks. From low-confidence
labels, we iteratively improve their quality by simultaneous learning of
machines and experts. We call it Human And Machine co-LEarning Technique
(HAMLET). Throughout the process, experts become more consistent, while the
algorithm provides them with explainable feedback for confirmation. HAMLET uses
a neural embedding function and a memory module filled with diverse reference
embeddings from different classes. Its output includes classification labels
and highly relevant reference embeddings as explanation. We took the study of
brain monitoring at intensive care unit (ICU) as an application of HAMLET on
continuous electroencephalography (cEEG) data. Although cEEG monitoring yields
large volumes of data, labeling costs and difficulty make it hard to build a
classifier. Additionally, while experts agree on the labels of clear-cut
examples of cEEG patterns, labeling many real-world cEEG data can be extremely
challenging. Thus, a large minority of sequences might be mislabeled. HAMLET
has shown significant performance gain against deep learning and other
baselines, increasing accuracy from 7.03% to 68.75% on challenging inputs.
Besides improved performance, clinical experts confirmed the interpretability
of those reference embeddings in helping explaining the classification results
by HAMLET.
