Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound

02/19/2021
by   Shion Takeno, et al.
0

Recently, several Bayesian optimization (BO) methods have been extended to the expensive black-box optimization problem with unknown constraints, which is an important problem that appears frequently in practice. We focus on an information-theoretic approach called Max-value Entropy Search (MES) whose superior performance has been repeatedly shown in BO literature. Since existing MES-based constrained BO is restricted to only one constraint, we first extend it to multiple constraints, but we found that this approach can cause negative approximate values for the mutual information, which can result in unreasonable decisions. In this paper, we employ a different approximation strategy that is based on a lower bound of the mutual information, and propose a novel constrained BO method called Constrained Max-value Entropy Search via Information lower BOund (CMES-IBO). Our approximate mutual information derived from the lower bound has a simple closed-form that is guaranteed to be nonnegative, and we show that irrational behavior caused by the negative value can be avoided. Furthermore, by using conditional mutual information, we extend our methods to the parallel setting in which multiple queries can be issued simultaneously. Finally, we demonstrate the effectiveness of our proposed methods by benchmark functions and real-world applications to materials science.

READ FULL TEXT
research
03/10/2023

Gaussian Max-Value Entropy Search for Multi-Agent Bayesian Optimization

We study the multi-agent Bayesian optimization (BO) problem, where multi...
research
02/28/2022

Rectified Max-Value Entropy Search for Bayesian Optimization

Although the existing max-value entropy search (MES) is based on the wid...
research
07/30/2021

A Training-Based Mutual Information Lower Bound for Large-Scale Systems

We provide a mutual information lower bound that can be used to analyze ...
research
04/11/2022

{PF}^2ES: Parallel Feasible Pareto Frontier Entropy Search for Multi-Objective Bayesian Optimization Under Unknown Constraints

We present Parallel Feasible Pareto Frontier Entropy Search ({PF}^2ES) –...
research
11/16/2020

Regularized Mutual Information Neural Estimation

With the variational lower bound of mutual information (MI), the estimat...
research
06/01/2022

Merlin-Arthur Classifiers: Formal Interpretability with Interactive Black Boxes

We present a new theoretical framework for making black box classifiers ...
research
04/01/2020

An approximate KLD based experimental design for models with intractable likelihoods

Data collection is a critical step in statistical inference and data sci...

Please sign up or login with your details

Forgot password? Click here to reset