BOCK : Bayesian Optimization with Cylindrical Kernels

06/05/2018
by   ChangYong Oh, et al.
4

A major challenge in Bayesian Optimization is the boundary issue (Swersky, 2017) where an algorithm spends too many evaluations near the boundary of its search space. In this paper, we propose BOCK, Bayesian Optimization with Cylindrical Kernels, whose basic idea is to transform the ball geometry of the search space using a cylindrical transformation. Because of the transformed geometry, the Gaussian Process-based surrogate model spends less budget searching near the boundary, while concentrating its efforts relatively more near the center of the search region, where we expect the solution to be located. We evaluate BOCK extensively, showing that it is not only more accurate and efficient, but it also scales successfully to problems with a dimensionality as high as 500. We show that the better accuracy and scalability of BOCK even allows optimizing modestly sized neural network layers, as well as neural network hyperparameters.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/18/2019

A Simple Heuristic for Bayesian Optimization with A Low Budget

The aim of black-box optimization is to optimize an objective function w...
research
04/17/2023

Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization

The linearized-Laplace approximation (LLA) has been shown to be effectiv...
research
04/04/2017

Correcting boundary over-exploration deficiencies in Bayesian optimization with virtual derivative sign observations

Bayesian optimization () is a global optimization strategy designed to f...
research
03/09/2018

Bayesian Optimization for Dynamic Problems

We propose practical extensions to Bayesian optimization for solving dyn...
research
11/02/2021

Geometry-aware Bayesian Optimization in Robotics using Riemannian Matérn Kernels

Bayesian optimization is a data-efficient technique which can be used fo...
research
02/18/2019

The Kalai-Smorodinski solution for many-objective Bayesian optimization

An ongoing aim of research in multiobjective Bayesian optimization is to...
research
12/06/2016

Factored Contextual Policy Search with Bayesian Optimization

Scarce data is a major challenge to scaling robot learning to truly comp...

Please sign up or login with your details

Forgot password? Click here to reset