Exploiting correlation and budget constraints in Bayesian multi-armed bandit optimization

03/27/2013
by   Matthew W. Hoffman, et al.
0

We address the problem of finding the maximizer of a nonlinear smooth function, that can only be evaluated point-wise, subject to constraints on the number of permitted function evaluations. This problem is also known as fixed-budget best arm identification in the multi-armed bandit literature. We introduce a Bayesian approach for this problem and show that it empirically outperforms both the existing frequentist counterpart and other Bayesian optimization methods. The Bayesian approach places emphasis on detailed modelling, including the modelling of correlations among the arms. As a result, it can perform well in situations where the number of arms is much larger than the number of allowed function evaluation, whereas the frequentist counterpart is inapplicable. This feature enables us to develop and deploy practical applications, such as automatic machine learning toolboxes. The paper presents comprehensive comparisons of the proposed approach, Thompson sampling, classical Bayesian optimization techniques, more recent Bayesian bandit approaches, and state-of-the-art best arm identification methods. This is the first comparison of many of these methods in the literature and allows us to examine the relative merits of their different features.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/14/2020

Bayesian Optimization – Multi-Armed Bandit Problem

In this report, we survey Bayesian Optimization methods focussed on the ...
research
11/28/2019

Bayesian Optimization for Categorical and Category-Specific Continuous Inputs

Many real-world functions are defined over both categorical and category...
research
01/17/2021

TSEC: a framework for online experimentation under experimental constraints

Thompson sampling is a popular algorithm for solving multi-armed bandit ...
research
06/05/2021

Multi-armed Bandit Algorithms on System-on-Chip: Go Frequentist or Bayesian?

Multi-armed Bandit (MAB) algorithms identify the best arm among multiple...
research
01/31/2017

Learning the distribution with largest mean: two bandit frameworks

Over the past few years, the multi-armed bandit model has become increas...
research
09/18/2019

Environmental Hotspot Identification in Limited Time with a UAV Equipped with a Downward-Facing Camera

We are motivated by environmental monitoring tasks where finding the glo...
research
06/20/2019

The Finite-Horizon Two-Armed Bandit Problem with Binary Responses: A Multidisciplinary Survey of the History, State of the Art, and Myths

In this paper we consider the two-armed bandit problem, which often natu...

Please sign up or login with your details

Forgot password? Click here to reset