The Sample Complexity of Best-k Items Selection from Pairwise Comparisons

07/06/2020
by   Wenbo Ren, et al.
4

This paper studies the sample complexity (aka number of comparisons) bounds for the active best-k items selection from pairwise comparisons. From a given set of items, the learner can make pairwise comparisons on every pair of items, and each comparison returns an independent noisy result about the preferred item. At any time, the learner can adaptively choose a pair of items to compare according to past observations (i.e., active learning). The learner's goal is to find the (approximately) best-k items with a given confidence, while trying to use as few comparisons as possible. In this paper, we study two problems: (i) finding the probably approximately correct (PAC) best-k items and (ii) finding the exact best-k items, both under strong stochastic transitivity and stochastic triangle inequality. For PAC best-k items selection, we first show a lower bound and then propose an algorithm whose sample complexity upper bound matches the lower bound up to a constant factor. For the exact best-k items selection, we first prove a worst-instance lower bound. We then propose two algorithms based on our PAC best items selection algorithms: one works for k=1 and is sample complexity optimal up to a loglog factor, and the other works for all values of k and is sample complexity optimal up to a log factor.

READ FULL TEXT
research
06/08/2018

PAC Ranking from Pairwise and Listwise Queries: Lower Bounds and Upper Bounds

This paper explores the adaptively (active) PAC (probably approximately ...
research
09/07/2019

On Sample Complexity Upper and Lower Bounds for Exact Ranking from Noisy Comparisons

This paper studies the problem of finding the exact ranking from noisy c...
research
06/12/2019

Sorted Top-k in Rounds

We consider the sorted top-k problem whose goal is to recover the top-k ...
research
03/14/2016

Active Algorithms For Preference Learning Problems with Multiple Populations

In this paper we model the problem of learning preferences of a populati...
research
04/10/2022

Active Learning with Label Comparisons

Supervised learning typically relies on manual annotation of the true la...
research
05/23/2016

Actively Learning Hemimetrics with Applications to Eliciting User Preferences

Motivated by an application of eliciting users' preferences, we investig...
research
04/24/2017

Learning from Comparisons and Choices

When tracking user-specific online activities, each user's preference is...

Please sign up or login with your details

Forgot password? Click here to reset