Single versus Double Blind Reviewing at WSDM 2017

02/01/2017
by   Andrew Tomkins, et al.
0

In this paper we study the implications for conference program committees of using single-blind reviewing, in which committee members are aware of the names and affiliations of paper authors, versus double-blind reviewing, in which this information is not visible to committee members. WSDM 2017, the 10th ACM International ACM Conference on Web Search and Data Mining, performed a controlled experiment in which each paper was reviewed by four committee members. Two of these four reviewers were chosen from a pool of committee members who had access to author information; the other two were chosen from a disjoint pool who did not have access to this information. This information asymmetry persisted through the process of bidding for papers, reviewing papers, and entering scores. Reviewers in the single-blind condition typically bid for 22 institutions. Once papers were allocated to reviewers, single-blind reviewers were significantly more likely than their double-blind counterparts to recommend for acceptance papers from famous authors and top institutions. The estimated odds multipliers are 1.63 for famous authors and 1.58 and 2.10 for top universities and companies respectively, so the result is tangible. For female authors, the associated odds multiplier of 0.78 is not statistically significant in our study. However, a meta-analysis places this value in line with that of other experiments, and in the context of this larger aggregate the gender effect is also statistically significant.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/06/2018

Uptake and outcome of manuscripts in Nature journals by review model and author characteristics

Double-blind peer review has been proposed as a possible solution to avo...
research
07/01/2020

De-anonymization of authors through arXiv submissions during double-blind review

In this paper, we investigate the effects of releasing arXiv preprints o...
research
11/29/2017

Proceedings 10th Interaction and Concurrency Experience

This volume contains the proceedings of ICE 2017, the 10th Interaction a...
research
09/05/2017

Effectiveness of Anonymization in Double-Blind Review

Double-blind review relies on the authors' ability and willingness to ef...
research
06/19/2023

The Myth of Meritocracy and the Matilda Effect in STEM: Paper Acceptance and Paper Citation

Biases against women in the workplace have been documented in various st...
research
03/31/2022

To ArXiv or not to ArXiv: A Study Quantifying Pros and Cons of Posting Preprints Online

Double-blind conferences have engaged in debates over whether to allow a...
research
05/02/2023

Multidimensional Fairness in Paper Recommendation

To prevent potential bias in the paper review and selection process for ...

Please sign up or login with your details

Forgot password? Click here to reset