DeepAI AI Chat
Log In Sign Up

Distributionally Robust Bayesian Optimization with φ-divergences

by   Hisham Husain, et al.

The study of robustness has received much attention due to its inevitability in data-driven settings where many systems face uncertainty. One such example of concern is Bayesian Optimization (BO), where uncertainty is multi-faceted, yet there only exists a limited number of works dedicated to this direction. In particular, there is the work of Kirschner et al. (2020), which bridges the existing literature of Distributionally Robust Optimization (DRO) by casting the BO problem from the lens of DRO. While this work is pioneering, it admittedly suffers from various practical shortcomings such as finite contexts assumptions, leaving behind the main question Can one devise a computationally tractable algorithm for solving this DRO-BO problem? In this work, we tackle this question to a large degree of generality by considering robustness against data-shift in ϕ-divergences, which subsumes many popular choices, such as the χ^2-divergence, Total Variation, and the extant Kullback-Leibler (KL) divergence. We show that the DRO-BO problem in this setting is equivalent to a finite-dimensional optimization problem which, even in the continuous context setting, can be easily implemented with provable sublinear regret bounds. We then show experimentally that our method surpasses existing methods, attesting to the theoretical results


page 6

page 15


Distributionally Robust Bayesian Optimization

Robustness to distributional shift is one of the key challenges of conte...

Tight Regret Bounds for Bayesian Optimization in One Dimension

We consider the problem of Bayesian optimization (BO) in one dimension, ...

High Dimensional Bayesian Optimization Using Dropout

Scaling Bayesian optimization to high dimensions is challenging task as ...

Distributional Robustness with IPMs and links to Regularization and GANs

Robustness to adversarial attacks is an important concern due to the fra...

Discretization-free Knowledge Gradient Methods for Bayesian Optimization

This paper studies Bayesian ranking and selection (R&S) problems with co...

Optimal design of photonic nanojets under uncertainty

Photonic nanojets (PNJs) have promising applications as optical probes i...