  We introduce a new scientific named entity recognizer called SEPT, which
stands for Span Extractor with Pre-trained Transformers. In recent papers, span
extractors have been demonstrated to be a powerful model compared with sequence
labeling models. However, we discover that with the development of pre-trained
language models, the performance of span extractors appears to become similar
to sequence labeling models. To keep the advantages of span representation, we
modified the model by under-sampling to balance the positive and negative
samples and reduce the search space. Furthermore, we simplify the origin
network architecture to combine the span extractor with BERT. Experiments
demonstrate that even simplified architecture achieves the same performance and
SEPT achieves a new state of the art result in scientific named entity
recognition even without relation information involved.
