High Dimensional Bayesian Optimisation and Bandits via Additive Models

03/05/2015
by   Kirthevasan Kandasamy, et al.
0

Bayesian Optimisation (BO) is a technique used in optimising a D-dimensional function which is typically expensive to evaluate. While there have been many successes for BO in low dimensions, scaling it to high dimensions has been notoriously difficult. Existing literature on the topic are under very restrictive settings. In this paper, we identify two key challenges in this endeavour. We tackle these challenges by assuming an additive structure for the function. This setting is substantially more expressive and contains a richer class of functions than previous work. We prove that, for additive functions the regret has only linear dependence on D even though the function depends on all D dimensions. We also demonstrate several other statistical and computational benefits in our framework. Via synthetic examples, a scientific simulation and a face detection problem we demonstrate that our method outperforms naive BO on additive functions and on several examples where the function is not additive.

READ FULL TEXT
research
05/31/2023

Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimization

Bayesian Optimization (BO) is typically used to optimize an unknown func...
research
07/21/2019

High Dimensional Bayesian Optimization via Supervised Dimension Reduction

Bayesian optimization (BO) has been broadly applied to computational exp...
research
02/20/2018

High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups

Bayesian optimization (BO) is a popular technique for sequential black-b...
research
06/21/2020

Additive Tree-Structured Covariance Function for Conditional Parameter Spaces in Bayesian Optimization

Bayesian optimization (BO) is a sample-efficient global optimization alg...
research
05/17/2022

High-dimensional additive Gaussian processes under monotonicity constraints

We introduce an additive Gaussian process framework accounting for monot...
research
01/31/2016

Additive Approximations in High Dimensional Nonparametric Regression via the SALSA

High dimensional nonparametric regression is an inherently difficult pro...
research
04/18/2016

Learning Sparse Additive Models with Interactions in High Dimensions

A function f: R^d →R is referred to as a Sparse Additive Model (SPAM), i...

Please sign up or login with your details

Forgot password? Click here to reset