Detecting Stance in Scientific Papers: Did we get more Negative Recently?

02/28/2022
by   Dominik Beese, et al.
1

In this paper, we classify scientific articles in the domain of natural language processing (NLP) and machine learning (ML) into whether (i) they extend the current state-of-the-art by introduction of novel techniques which beat existing models or whether (ii) they mainly criticize the existing state-of-the-art, i.e., that it is deficient with respect to some property (e.g., wrong evaluation, wrong datasets, misleading task specification). We refer to contributions under (i) as having a "positive stance" and contributions under (ii) as having a "negative stance" to related work. We annotate over 2k papers from NLP and ML to train a SciBERT based model to automatically predict the stance of a paper based on its title and abstract. We then analyze large-scale trends on over 41k papers from the last  35 years in NLP and ML, finding that papers have gotten substantially more positive over time, but negative papers also got more negative and we observe considerably more negative papers in recent years. Negative papers are also more influential in terms of citations they receive.

READ FULL TEXT
research
05/02/2020

Examining Citations of Natural Language Processing Literature

We extracted information from the ACL Anthology (AA) and Google Scholar ...
research
07/31/2023

NLLG Quarterly arXiv Report 06/23: What are the most influential current AI Papers?

The rapid growth of information in the field of Generative Artificial In...
research
11/30/2022

A Major Obstacle for NLP Research: Let's Talk about Time Allocation!

The field of natural language processing (NLP) has grown over the last f...
research
06/06/2021

How Did This Get Funded?! Automatically Identifying Quirky Scientific Achievements

Humor is an important social phenomenon, serving complex social and psyc...
research
11/04/2022

NLP Inspired Training Mechanics For Modeling Transient Dynamics

In recent years, Machine learning (ML) techniques developed for Natural ...
research
02/10/2018

Distributed NLP

In this paper we present the performance of parallel text processing wit...
research
02/23/2023

Natural Language Processing in the Legal Domain

In this paper, we summarize the current state of the field of NLP La...

Please sign up or login with your details

Forgot password? Click here to reset