Optimal Clustering from Noisy Binary Feedback

by   Kaito Ariu, et al.

We study the problem of recovering clusters from binary user feedback. Items are grouped into initially unknown non-overlapping clusters. To recover these clusters, the learner sequentially presents to users a finite list of items together with a question with a binary answer selected from a fixed finite set. For each of these items, the user provides a random answer whose expectation is determined by the item cluster and the question and by an item-specific parameter characterizing the hardness of classifying the item. The objective is to devise an algorithm with a minimal cluster recovery error rate. We derive problem-specific information-theoretical lower bounds on the error rate satisfied by any algorithm, for both uniform and adaptive (list, question) selection strategies. For uniform selection, we present a simple algorithm built upon K-means whose performance almost matches the fundamental limits. For adaptive selection, we develop an adaptive algorithm that is inspired by the derivation of the information-theoretical error lower bounds, and in turn allocates the budget in an efficient way. The algorithm learns to select items hard to cluster and relevant questions more often. We compare numerically the performance of our algorithms with or without adaptive selection strategy, and illustrate the gain achieved by being adaptive. Our inference problems are motivated by the problem of solving large-scale labeling tasks with minimal effort put on the users. For example, in some of the recent CAPTCHA systems, users clicks (binary answers) can be used to efficiently label images, by optimally finding the best questions to present.


page 1

page 2

page 3

page 4


Regret in Online Recommendation Systems

This paper proposes a theoretical analysis of recommendation systems in ...

Discriminative Learning via Adaptive Questioning

We consider the problem of designing an adaptive sequence of questions t...

Collaborative Filtering with Information-Rich and Information-Sparse Entities

In this paper, we consider a popular model for collaborative filtering i...

Optimal Cluster Recovery in the Labeled Stochastic Block Model

We consider the problem of community detection or clustering in the labe...

Same-Cluster Querying for Overlapping Clusters

Overlapping clusters are common in models of many practical data-segment...

Clustering via Content-Augmented Stochastic Blockmodels

Much of the data being created on the web contains interactions between ...

Apple Tasting Revisited: Bayesian Approaches to Partially Monitored Online Binary Classification

We consider a variant of online binary classification where a learner se...