Inverse Bayesian Optimization: Learning Human Search Strategies in a Sequential Optimization Task

04/16/2021
by   Nathan Sandholtz, et al.
0

Bayesian optimization is a popular algorithm for sequential optimization of a latent objective function when sampling from the objective is costly. The search path of the algorithm is governed by the acquisition function, which defines the agent's search strategy. Conceptually, the acquisition function characterizes how the optimizer balances exploration and exploitation when searching for the optimum of the latent objective. In this paper, we explore the inverse problem of Bayesian optimization; we seek to estimate the agent's latent acquisition function based on observed search paths. We introduce a probabilistic solution framework for the inverse problem which provides a principled framework to quantify both the variability with which the agent performs the optimization task as well as the uncertainty around their estimated acquisition function. We illustrate our methods by analyzing human behavior from an experiment which was designed to force subjects to balance exploration and exploitation in search of an invisible target location. We find that while most subjects demonstrate clear trends in their search behavior, there is significant variation around these trends from round to round. A wide range of search strategies are exhibited across the subjects in our study, but upper confidence bound acquisition functions offer the best fit for the majority of subjects. Finally, some subjects do not map well to any of the acquisition functions we initially consider; these subjects tend to exhibit exploration preferences beyond that of standard acquisition functions to capture. Guided by the model discrepancies, we augment the candidate acquisition functions to yield a superior fit to the human behavior in this task.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/24/2019

On Local Optimizers of Acquisition Functions in Bayesian Optimization

Bayesian optimization is a sample-efficient method for finding a global ...
research
04/28/2016

A Probabilistic Adaptive Search System for Exploring the Face Space

Face recall is a basic human cognitive process performed routinely, e.g....
research
10/18/2021

Efficient Exploration in Binary and Preferential Bayesian Optimization

Bayesian optimization (BO) is an effective approach to optimize expensiv...
research
03/30/2012

A Lipschitz Exploration-Exploitation Scheme for Bayesian Optimization

The problem of optimizing unknown costly-to-evaluate functions has been ...
research
09/01/2023

Polynomial-Model-Based Optimization for Blackbox Objectives

For a wide range of applications the structure of systems like Neural Ne...
research
10/29/2011

Bayesian Optimization for Adaptive MCMC

This paper proposes a new randomized strategy for adaptive MCMC using Ba...

Please sign up or login with your details

Forgot password? Click here to reset