-
HEBO: Heteroscedastic Evolutionary Bayesian Optimisation
We introduce HEBO: Heteroscedastic Evolutionary Bayesian Optimisation th...
read it
-
Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly
Bayesian Optimisation (BO), refers to a suite of techniques for global o...
read it
-
Incorporating Expert Prior in Bayesian Optimisation via Space Warping
Bayesian optimisation is a well-known sample-efficient method for the op...
read it
-
Emergent Language Generalization and Acquisition Speed are not tied to Compositionality
Studies of discrete languages emerging when neural agents communicate to...
read it
-
Compositional ADAM: An Adaptive Compositional Solver
In this paper, we present C-ADAM, the first adaptive solver for composit...
read it
-
The reparameterization trick for acquisition functions
Bayesian optimization is a sample-efficient approach to solving global o...
read it
-
Greed is Good: Exploration and Exploitation Trade-offs in Bayesian Optimisation
The performance of acquisition functions for Bayesian optimisation is in...
read it
Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?
Bayesian optimisation presents a sample-efficient methodology for global optimisation. Within this framework, a crucial performance-determining subroutine is the maximisation of the acquisition function, a task complicated by the fact that acquisition functions tend to be non-convex and thus nontrivial to optimise. In this paper, we undertake a comprehensive empirical study of approaches to maximise the acquisition function. Additionally, by deriving novel, yet mathematically equivalent, compositional forms for popular acquisition functions, we recast the maximisation task as a compositional optimisation problem, allowing us to benefit from the extensive literature in this field. We highlight the empirical advantages of the compositional approach to acquisition function maximisation across 3958 individual experiments comprising synthetic optimisation tasks as well as tasks from Bayesmark. Given the generality of the acquisition function maximisation subroutine, we posit that the adoption of compositional optimisers has the potential to yield performance improvements across all domains in which Bayesian optimisation is currently being applied.
READ FULL TEXT
Comments
There are no comments yet.