Nearly Optimal Sampling Algorithms for Combinatorial Pure Exploration

06/04/2017
by   Lijie Chen, et al.
0

We study the combinatorial pure exploration problem Best-Set in stochastic multi-armed bandits. In a Best-Set instance, we are given n arms with unknown reward distributions, as well as a family F of feasible subsets over the arms. Our goal is to identify the feasible subset in F with the maximum total mean using as few samples as possible. The problem generalizes the classical best arm identification problem and the top-k arm identification problem, both of which have attracted significant attention in recent years. We provide a novel instance-wise lower bound for the sample complexity of the problem, as well as a nontrivial sampling algorithm, matching the lower bound up to a factor of |F|. For an important class of combinatorial families, we also provide polynomial time implementation of the sampling algorithm, using the equivalence of separation and optimization for convex program, and approximate Pareto curves in multi-objective optimization. We also show that the |F| factor is inevitable in general through a nontrivial lower bound construction. Our results significantly improve several previous results for several important combinatorial constraints, and provide a tighter understanding of the general Best-Set problem. We further introduce an even more general problem, formulated in geometric terms. We are given n Gaussian arms with unknown means and unit variance. Consider the n-dimensional Euclidean space R^n, and a collection O of disjoint subsets. Our goal is to determine the subset in O that contains the n-dimensional vector of the means. The problem generalizes most pure exploration bandit problems studied in the literature. We provide the first nearly optimal sample complexity upper and lower bounds for the problem.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/04/2018

Combinatorial Pure Exploration with Continuous and Separable Reward Functions and Its Applications (Extended Version)

We study the Combinatorial Pure Exploration problem with Continuous and ...
research
05/19/2017

Practical Algorithms for Best-K Identification in Multi-Armed Bandits

In the Best-K identification problem (Best-K-Arm), we are given N stocha...
research
02/03/2023

An Asymptotically Optimal Algorithm for the One-Dimensional Convex Hull Feasibility Problem

This work studies the pure-exploration setting for the convex hull feasi...
research
11/21/2017

Disagreement-based combinatorial pure exploration: Efficient algorithms and an analysis with localization

We design new algorithms for the combinatorial pure exploration problem ...
research
02/24/2021

Combinatorial Pure Exploration with Bottleneck Reward Function and its Extension to General Reward Functions

In this paper, we study the Combinatorial Pure Exploration problem with ...
research
06/15/2023

Combinatorial Pure Exploration of Multi-Armed Bandit with a Real Number Action Class

The combinatorial pure exploration (CPE) in the stochastic multi-armed b...
research
12/08/2021

A Fast Algorithm for PAC Combinatorial Pure Exploration

We consider the problem of Combinatorial Pure Exploration (CPE), which d...

Please sign up or login with your details

Forgot password? Click here to reset