Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly

by   Kirthevasan Kandasamy, et al.

Bayesian Optimisation (BO), refers to a suite of techniques for global optimisation of expensive black box functions, which use introspective Bayesian models of the function to efficiently find the optimum. While BO has been applied successfully in many applications, modern optimisation tasks usher in new challenges where conventional methods fail spectacularly. In this work, we present Dragonfly, an open source Python library for scalable and robust BO. Dragonfly incorporates multiple recently developed methods that allow BO to be applied in challenging real world settings; these include better methods for handling higher dimensional domains, methods for handling multi-fidelity evaluations when cheap approximations of an expensive function are available, methods for optimising over structured combinatorial spaces, such as the space of neural network architectures, and methods for handling parallel evaluations. Additionally, we develop new methodological improvements in BO for selecting the Bayesian model, selecting the acquisition function, and optimising over complex domains with different variable types and additional constraints. We compare Dragonfly to a suite of other packages and algorithms for global optimisation and demonstrate that when the above methods are integrated, they enable significant improvements in the performance of BO. The Dragonfly library is available at dragonfly.github.io.


page 1

page 2

page 3

page 4


Multi-fidelity Bayesian Optimisation with Continuous Approximations

Bandit methods for black-box optimisation, such as Bayesian optimisation...

Incorporating Expert Prior in Bayesian Optimisation via Space Warping

Bayesian optimisation is a well-known sample-efficient method for the op...

Cost-aware Multi-objective Bayesian optimisation

The notion of expense in Bayesian optimisation generally refers to the u...

Bayesian Optimisation with Formal Guarantees

Application domains of Bayesian optimization include optimizing black-bo...

Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?

Bayesian optimisation presents a sample-efficient methodology for global...

Asynchronous Parallel Bayesian Optimisation via Thompson Sampling

We design and analyse variations of the classical Thompson sampling (TS)...

Multi-fidelity Gaussian Process Bandit Optimisation

In many scientific and engineering applications, we are tasked with the ...

Code Repositories


An open source python library for scalable Bayesian optimisation.

view repo