Argument Mining Driven Analysis of Peer-Reviews

12/10/2020
by   Michael Fromm, et al.
0

Peer reviewing is a central process in modern research and essential for ensuring high quality and reliability of published work. At the same time, it is a time-consuming process and increasing interest in emerging fields often results in a high review workload, especially for senior researchers in this area. How to cope with this problem is an open question and it is vividly discussed across all major conferences. In this work, we propose an Argument Mining based approach for the assistance of editors, meta-reviewers, and reviewers. We demonstrate that the decision process in the field of scientific publications is driven by arguments and automatic argument identification is helpful in various use-cases. One of our findings is that arguments used in the peer-review process differ from arguments in other domains making the transfer of pre-trained models difficult. Therefore, we provide the community with a new peer-review dataset from different computer science conferences with annotated arguments. In our extensive empirical evaluation, we show that Argument Mining can be used to efficiently extract the most relevant parts from reviews, which are paramount for the publication decision. The process remains interpretable since the extracted arguments can be highlighted in a review without detaching them from their context.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/25/2019

Argument Mining for Understanding Peer Reviews

Peer-review plays a critical role in the scientific writing and publicat...
research
07/23/2017

Using Argument-based Features to Predict and Analyse Review Helpfulness

We study the helpful product reviews identification problem in this pape...
research
08/27/2021

Auctions and Prediction Markets for Scientific Peer Review

Peer reviewed publications are considered the gold standard in certifyin...
research
04/22/2022

Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review

Peer review is a key component of the publishing process in most fields ...
research
12/03/2018

Automatically Annotating Articles Towards Opening and Reusing Transparent Peer Reviews

An increasing number of scientific publications are created in open and ...
research
03/31/2020

State-of-Art-Reviewing: A Radical Proposal to Improve Scientific Publication

Peer review forms the backbone of modern scientific manuscript evaluatio...
research
03/29/2022

Empirical Standards for Repository Mining

The purpose of scholarly peer review is to evaluate the quality of scien...

Please sign up or login with your details

Forgot password? Click here to reset