  This paper studies the end-to-end construction of an NLP Knowledge Graph (KG)
from scientific papers. We focus on extracting four types of relations:
evaluatedOn between tasks and datasets, evaluatedBy between tasks and
evaluation metrics, as well as coreferent and related relations between the
same type of entities. For instance, F1-score is coreferent with F-measure. We
introduce novel methods for each of these relation types and apply our final
framework (SciNLP-KG) to 30,000 NLP papers from ACL Anthology to build a
large-scale KG, which can facilitate automatically constructing scientific
leaderboards for the NLP community. The results of our experiments indicate
that the resulting KG contains high-quality information.
