Discriminative Learning via Adaptive Questioning

04/11/2020
by   Achal Bassamboo, et al.
0

We consider the problem of designing an adaptive sequence of questions that optimally classify a candidate's ability into one of several categories or discriminative grades. A candidate's ability is modeled as an unknown parameter, which, together with the difficulty of the question asked, determines the likelihood with which s/he is able to answer a question correctly. The learning algorithm is only able to observe these noisy responses to its queries. We consider this problem from a fixed confidence-based δ-correct framework, that in our setting seeks to arrive at the correct ability discrimination at the fastest possible rate while guaranteeing that the probability of error is less than a pre-specified and small δ. In this setting we develop lower bounds on any sequential questioning strategy and develop geometrical insights into the problem structure both from primal and dual formulation. In addition, we arrive at algorithms that essentially match these lower bounds. Our key conclusions are that, asymptotically, any candidate needs to be asked questions at most at two (candidate ability-specific) levels, although, in a reasonably general framework, questions need to be asked only at a single level. Further, and interestingly, the problem structure facilitates endogenous exploration, so there is no need for a separately designed exploration stage in the algorithm.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/21/2023

On the Optimal Bounds for Noisy Computing

We revisit the problem of computing with noisy information considered in...
research
10/14/2019

Optimal Clustering from Noisy Binary Feedback

We study the problem of recovering clusters from binary user feedback. I...
research
06/05/2023

Active Ranking of Experts Based on their Performances in Many Tasks

We consider the problem of ranking n experts based on their performances...
research
08/24/2019

Optimal best arm selection for general distributions

Given a finite set of unknown distributions or arms that can be sampled ...
research
10/29/2017

Improved Bounds for Testing Forbidden Order Patterns

A sequence f{1,...,n}→R contains a permutation π of length k if there ex...
research
07/05/2021

Memory-Sample Lower Bounds for Learning Parity with Noise

In this work, we show, for the well-studied problem of learning parity u...

Please sign up or login with your details

Forgot password? Click here to reset