Accelerated Experimental Design for Pairwise Comparisons

01/18/2019
by   Yuan Guo, et al.
0

Pairwise comparison labels are more informative and less variable than class labels, but generating them poses a challenge: their number grows quadratically in the dataset size. We study a natural experimental design objective, namely, D-optimality, that can be used to identify which K pairwise comparisons to generate. This objective is known to perform well in practice, and is submodular, making the selection approximable via the greedy algorithm. A naïve greedy implementation has O(N^2d^2K) complexity, where N is the dataset size, d is the feature space dimension, and K is the number of generated comparisons. We show that, by exploiting the inherent geometry of the dataset--namely, that it consists of pairwise comparisons--the greedy algorithm's complexity can be reduced to O(N^2(K+d)+N(dK+d^2) +d^2K). We apply the same acceleration also to the so-called lazy greedy algorithm. When combined, the above improvements lead to an execution time of less than 1 hour for a dataset with 10^8 comparisons; the naïve greedy algorithm on the same dataset would require more than 10 days to terminate.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/05/2018

Scaling Submodular Optimization Approaches for Control Applications in Networked Systems

Often times, in many design problems, there is a need to select a small ...
research
03/28/2015

Recent advances on inconsistency indices for pairwise comparisons - a commentary

This paper recalls the definition of consistency for pairwise comparison...
research
05/20/2015

Managing Null Entries in Pairwise Comparisons

This paper shows how to manage null entries in pairwise comparisons matr...
research
06/09/2019

Aggregation of pairwise comparisons with reduction of biases

We study the problem of ranking from crowdsourced pairwise comparisons. ...
research
06/28/2017

Submodular Function Maximization for Group Elevator Scheduling

We propose a novel approach for group elevator scheduling by formulating...
research
09/12/2015

Axiomatization of Inconsistency Indicators for Pairwise Comparisons

This study proposes revised axioms for defining inconsistency indicators...
research
08/03/2020

Classification from Ambiguity Comparisons

Labeling data is an unavoidable pre-processing procedure for most machin...

Please sign up or login with your details

Forgot password? Click here to reset