A Community's Perspective on the Status and Future of Peer Review in Software Engineering

06/22/2017
by   Lutz Prechelt, et al.
0

Context: Pre-publication peer review of scientific articles is considered a key element of the research process in software engineering, yet it is often perceived as not to work fully well. Objective: We aim at understanding the perceptions of and attitudes towards peer review of authors and reviewers at one of software engineering's most prestigious venues, the International Conference on Software Engineering (ICSE). Method: We invited 932 ICSE 2014/15/16 authors and reviewers to participate in a survey with 10 closed and 9 open questions. Results: We present a multitude of results, such as: Respondents perceive only one third of all reviews to be good, yet one third as useless or misleading; they propose double-blind or zero-blind reviewing regimes for improvement; they would like to see showable proofs of (good) reviewing work be introduced; attitude change trends are weak. Conclusion: The perception of the current state of software engineering peer review is fairly negative. Also, we found hardly any trend that suggests reviewing will improve by itself over time; the community will have to make explicit efforts. Fortunately, our (mostly senior) respondents appear more open for trying different peer reviewing regimes than we had expected.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/02/2020

Understanding Peer Review of Software Engineering Papers

Peer review is a key activity intended to preserve the quality and integ...
research
10/07/2020

Empirical Standards for Software Engineering Research

Empirical Standards are natural-language models of a scientific communit...
research
07/04/2018

Case for the double-blind peer review

Peer review is a process designed to produce a fair assessment of resear...
research
11/14/2022

Cracking Double-Blind Review: Authorship Attribution with Deep Learning

Double-blind peer review is considered a pillar of academic research bec...
research
10/12/2019

Efficiency in managing peer-review of scientific manuscripts – editors' perspective

The purpose of this paper is to introduce a model for measuring the effi...
research
05/02/2022

What Factors Should Paper-Reviewer Assignments Rely On? Community Perspectives on Issues and Ideals in Conference Peer-Review

Both scientific progress and individual researcher careers depend on the...
research
03/31/2020

State-of-Art-Reviewing: A Radical Proposal to Improve Scientific Publication

Peer review forms the backbone of modern scientific manuscript evaluatio...

Please sign up or login with your details

Forgot password? Click here to reset