A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications

04/25/2018
by   Dongyeop Kang, et al.
0

Peer reviewing is a central component in the scientific publishing process. We present the first public dataset of scientific peer reviews available for research purposes (PeerRead v1) providing an opportunity to study this important artifact. The dataset consists of 14.7K paper drafts and the corresponding accept/reject decisions in top-tier venues including ACL, NIPS and ICLR. The dataset also includes 10.7K textual peer reviews written by experts for a subset of the papers. We describe the data collection process and report interesting observed phenomena in the peer reviews. We also propose two novel NLP tasks based on this dataset and provide simple baseline models. In the first task, we show that simple models can predict whether a paper is accepted with up to 21 the second task, we predict the numerical scores of review aspects and show that simple models can outperform the mean baseline for aspects with high variance such as 'originality' and 'impact'.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/30/2021

Can We Automate Scientific Reviewing?

The rapid development of science and technology has been accompanied by ...
research
08/09/2021

COMPARE: A Taxonomy and Dataset of Comparison Discussions in Peer Reviews

Comparing research papers is a conventional method to demonstrate progre...
research
09/02/2021

Ranking Scientific Papers Using Preference Learning

Peer review is the main quality control mechanism in academia. Quality o...
research
06/05/2020

Aspect-based Sentiment Analysis of Scientific Reviews

Scientific papers are complex and understanding the usefulness of these ...
research
01/27/2022

Yes-Yes-Yes: Donation-based Peer Reviewing Data Collection for ACL Rolling Review and Beyond

Peer review is the primary gatekeeper of scientific merit and quality, y...
research
03/27/2019

Does My Rebuttal Matter? Insights from a Major NLP Conference

Peer review is a core element of the scientific process, particularly in...
research
06/21/2023

ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews

Revising scientific papers based on peer feedback is a challenging task ...

Please sign up or login with your details

Forgot password? Click here to reset