Additive Tree-Structured Covariance Function for Conditional Parameter Spaces in Bayesian Optimization

06/21/2020
by   Xingchen Ma, et al.
0

Bayesian optimization (BO) is a sample-efficient global optimization algorithm for black-box functions which are expensive to evaluate. Existing literature on model based optimization in conditional parameter spaces are usually built on trees. In this work, we generalize the additive assumption to tree-structured functions and propose an additive tree-structured covariance function, showing improved sample-efficiency, wider applicability and greater flexibility. Furthermore, by incorporating the structure information of parameter spaces and the additive assumption in the BO loop, we develop a parallel algorithm to optimize the acquisition function and this optimization can be performed in a low dimensional space. We demonstrate our method on an optimization benchmark function, as well as on a neural network model compression problem, and experimental results show our approach significantly outperforms the current state of the art for conditional parameter optimization including SMAC, TPE and Jenatton et al. (2017).

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/06/2020

Additive Tree-Structured Conditional Parameter Spaces in Bayesian Optimization: A Novel Covariance Function and a Fast Implementation

Bayesian optimization (BO) is a sample-efficient global optimization alg...
research
12/24/2020

High-Dimensional Bayesian Optimization via Tree-Structured Additive Models

Bayesian Optimization (BO) has shown significant success in tackling exp...
research
05/31/2023

Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimization

Bayesian Optimization (BO) is typically used to optimize an unknown func...
research
02/20/2018

High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups

Bayesian optimization (BO) is a popular technique for sequential black-b...
research
03/05/2015

High Dimensional Bayesian Optimisation and Bandits via Additive Models

Bayesian Optimisation (BO) is a technique used in optimising a D-dimensi...
research
02/26/2019

Topological Bayesian Optimization with Persistence Diagrams

Finding an optimal parameter of a black-box function is important for se...
research
10/21/2020

High-Dimensional Bayesian Optimization via Nested Riemannian Manifolds

Despite the recent success of Bayesian optimization (BO) in a variety of...

Please sign up or login with your details

Forgot password? Click here to reset