Joint aggregation of cardinal and ordinal evaluations with an application to a student paper competition

01/12/2021
by   Dorit S. Hochbaum, et al.
2

An important problem in decision theory concerns the aggregation of individual rankings/ratings into a collective evaluation. We illustrate a new aggregation method in the context of the 2007 MSOM's student paper competition. The aggregation problem in this competition poses two challenges. Firstly, each paper was reviewed only by a very small fraction of the judges; thus the aggregate evaluation is highly sensitive to the subjective scales chosen by the judges. Secondly, the judges provided both cardinal and ordinal evaluations (ratings and rankings) of the papers they reviewed. The contribution here is a new robust methodology that jointly aggregates ordinal and cardinal evaluations into a collective evaluation. This methodology is particularly suitable in cases of incomplete evaluations – i.e., when the individuals evaluate only a strict subset of the objects. This approach is potentially useful in managerial decision making problems by a committee selecting projects from a large set or capital budgeting involving multiple priorities.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/23/2023

Modeling Preferences: A Bayesian Mixture of Finite Mixtures for Rankings and Ratings

Rankings and ratings are commonly used to express preferences but provid...
research
11/17/2014

Aggregating partial rankings with applications to peer grading in massive online open courses

We investigate the potential of using ordinal peer grading for the evalu...
research
02/25/2016

How effective can simple ordinal peer grading be?

Ordinal peer grading has been proposed as a simple and scalable solution...
research
04/24/2017

Using a new parsimonious AHP methodology combined with the Choquet integral: An application for evaluating social housing initiatives

We propose a development of the Analytic Hierarchy Process (AHP) permitt...
research
04/03/2022

Meta-Learning Approaches for a One-Shot Collective-Decision Aggregation: Correctly Choosing how to Choose Correctly

Aggregating successfully the choices regarding a given decision problem ...
research
12/01/2021

Collective discrete optimisation as judgment aggregation

Many important collective decision-making problems can be seen as multi-...
research
07/06/2020

Machine Learning with the Sugeno Integral: The Case of Binary Classification

In this paper, we elaborate on the use of the Sugeno integral in the con...

Please sign up or login with your details

Forgot password? Click here to reset