  In this paper, we classify scientific articles in the domain of natural
language processing (NLP) and machine learning (ML), as core subfields of
artificial intelligence (AI), into whether (i) they extend the current
state-of-the-art by the introduction of novel techniques which beat existing
models or whether (ii) they mainly criticize the existing state-of-the-art,
i.e. that it is deficient with respect to some property (e.g. wrong evaluation,
wrong datasets, misleading task specification). We refer to contributions under
(i) as having a 'positive stance' and contributions under (ii) as having a
'negative stance' (to related work). We annotate over 1.5 k papers from NLP and
ML to train a SciBERT-based model to automatically predict the stance of a
paper based on its title and abstract. We then analyse large-scale trends on
over 41 k papers from the last approximately 35 years in NLP and ML, finding
that papers have become substantially more positive over time, but negative
papers also got more negative and we observe considerably more negative papers
in recent years. Negative papers are also more influential in terms of
citations they receive.
