Strategyproof Peer Selection using Randomization, Partitioning, and Apportionment

04/13/2016
by   Haris Aziz, et al.
0

Peer review, evaluation, and selection is a fundamental aspect of modern science. Funding bodies the world over employ experts to review and select the best proposals of those submitted for funding. The problem of peer selection, however, is much more general: a professional society may want to give a subset of its members awards based on the opinions of all members; an instructor for a MOOC or online course may want to crowdsource grading; or a marketing company may select ideas from group brainstorming sessions based on peer evaluation. We make three fundamental contributions to the study of procedures or mechanisms for peer selection, a specific type of group decision-making problem, studied in computer science, economics, and political science. First, we propose a novel mechanism that is strategyproof, i.e., agents cannot benefit by reporting insincere valuations. Second, we demonstrate the effectiveness of our mechanism by a comprehensive simulation-based comparison with a suite of mechanisms found in the literature. Finally, our mechanism employs a randomized rounding technique that is of independent interest, as it solves the apportionment problem that arises in various settings where discrete resources such as parliamentary representation slots need to be divided proportionally.

READ FULL TEXT
research
01/12/2020

Leveraging Peer Feedback to Improve Visualization Education

Peer review is a widely utilized pedagogical feedback mechanism for enga...
research
04/30/2020

PeerNomination: Relaxing Exactness for Increased Accuracy in Peer Selection

In peer selection agents must choose a subset of themselves for an award...
research
07/21/2021

Peer Selection with Noisy Assessments

In the peer selection problem a group of agents must select a subset of ...
research
10/05/2022

Manipulation and Peer Mechanisms: A Survey

In peer mechanisms, the competitors for a prize also determine who wins....
research
02/19/2021

Rethinking the Funding Line at the Swiss National Science Foundation: Bayesian Ranking and Lottery

Funding agencies rely on peer review and expert panels to select the res...
research
09/12/2013

Inducing Honest Reporting Without Observing Outcomes: An Application to the Peer-Review Process

When eliciting opinions from a group of experts, traditional devices use...
research
12/30/2019

Wisdom of collaborators: a peer-review approach to performance appraisal

Individual performance and reputation within a company are major factors...

Please sign up or login with your details

Forgot password? Click here to reset