Parameter Optimization using high-dimensional Bayesian Optimization

by   David Yenicelik, et al.

In this thesis, I explore the possibilities of conducting Bayesian optimization techniques in high dimensional domains. Although high dimensional domains can be defined to be between hundreds and thousands of dimensions, we will primarily focus on problem settings that occur between two and 20 dimensions. As such, we focus on solutions to practical problems, such as tuning the parameters for an electron accelerator, or for even simpler tasks that can be run and optimized just in time with a standard laptop at hand. Our main contributions are 1.) comparing how the log-likelihood affects the angle-difference in the real projection matrix, and the found matrix matrix, 2.) an extensive analysis of current popular methods including strengths and shortcomings, 3.) a short analysis on how dimensionality reduction techniques can be used for feature selection, and 4.) a novel algorithm called "BORING", which allows for a simple fallback mechanism if the matrix identification fails, as well as taking into consideration "passive" subspaces which provide small perturbations of the function at hand. The main features of BORING are 1.) the possibility to identify the subspace (unlike most other optimization algorithms), and 2.) to provide a much lower penalty to identify the subspace if identification fails, as optimization is still the primary goal.


page 1

page 2

page 3

page 4


Efficient and Scalable Batch Bayesian Optimization Using K-Means

We present K-Means Batch Bayesian Optimization (KMBBO), a novel batch sa...

High-Dimensional Bayesian Optimization with Sparse Axis-Aligned Subspaces

Bayesian optimization (BO) is a powerful paradigm for efficient optimiza...

Scaling Gaussian Process Regression with Derivatives

Gaussian processes (GPs) with derivatives are useful in many application...

Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces

Bayesian optimization is known to be difficult to scale to high dimensio...

Out-Of-Distribution Detection With Subspace Techniques And Probabilistic Modeling Of Features

This paper presents a principled approach for detecting out-of-distribut...

High-Dimensional Bayesian Optimization with Multi-Task Learning for RocksDB

RocksDB is a general-purpose embedded key-value store used in multiple d...

Bayesian Optimization in a Billion Dimensions via Random Embeddings

Bayesian optimization techniques have been successfully applied to robot...

Please sign up or login with your details

Forgot password? Click here to reset