  Existing models for named entity recognition (NER) are mainly based on
large-scale labeled datasets, which always obtain using crowdsourcing. However,
it is hard to obtain a unified and correct label via majority voting from
multiple annotators for NER due to the large labeling space and complexity of
this task. To address this problem, we aim to utilize the original
multi-annotator labels directly. Particularly, we propose a Confidence-based
Partial Label Learning (CPLL) method to integrate the prior confidence (given
by annotators) and posterior confidences (learned by models) for
crowd-annotated NER. This model learns a token- and content-dependent
confidence via an Expectation-Maximization (EM) algorithm by minimizing
empirical risk. The true posterior estimator and confidence estimator perform
iteratively to update the true posterior and confidence respectively. We
conduct extensive experimental results on both real-world and synthetic
datasets, which show that our model can improve performance effectively
compared with strong baselines.
