Making Paper Reviewing Robust to Bid Manipulation Attacks

by   Ruihan Wu, et al.

Most computer science conferences rely on paper bidding to assign reviewers to papers. Although paper bidding enables high-quality assignments in days of unprecedented submission numbers, it also opens the door for dishonest reviewers to adversarially influence paper reviewing assignments. Anecdotal evidence suggests that some reviewers bid on papers by "friends" or colluding authors, even though these papers are outside their area of expertise, and recommend them for acceptance without considering the merit of the work. In this paper, we study the efficacy of such bid manipulation attacks and find that, indeed, they can jeopardize the integrity of the review process. We develop a novel approach for paper bidding and assignment that is much more robust against such attacks. We show empirically that our approach provides robustness even when dishonest reviewers collude, have full knowledge of the assignment system's internal workings, and have access to the system's inputs. In addition to being more robust, the quality of our paper review assignments is comparable to that of current, non-robust assignment approaches.


Combating Collusion Rings is Hard but Possible

A recent report of Littmann [Commun. ACM '21] outlines the existence and...

Diverse Reviewer Suggestion for Extending Conference Program Committees

Automated reviewer recommendation for scientific conferences currently r...

Mitigating Manipulation in Peer Review via Randomized Reviewer Assignments

We consider three important challenges in conference peer review: (i) re...

Tradeoffs in Preventing Manipulation in Paper Bidding for Reviewer Assignment

Many conferences rely on paper bidding as a key component of their revie...

No more Reviewer #2: Subverting Automatic Paper-Reviewer Assignment using Adversarial Learning

The number of papers submitted to academic conferences is steadily risin...

A Gold Standard Dataset for the Reviewer Assignment Problem

Many peer-review venues are either using or looking to use algorithms to...

Please sign up or login with your details

Forgot password? Click here to reset