  Named Entity Recognition (NER) is a fundamental Natural Language Processing
(NLP) task to extract entities from unstructured data. The previous methods for
NER were based on machine learning or deep learning. Recently, pre-training
models have significantly improved performance on multiple NLP tasks. In this
paper, firstly, we introduce the architecture and pre-training tasks of four
common pre-training models: BERT, ERNIE, ERNIE2.0-tiny, and RoBERTa. Then, we
apply these pre-training models to a NER task by fine-tuning, and compare the
effects of the different model architecture and pre-training tasks on the NER
task. The experiment results showed that RoBERTa achieved state-of-the-art
results on the MSRA-2006 dataset.
