DeepAI AI Chat
Log In Sign Up

A comparison of mixed-variables Bayesian optimization approaches

by   Jhouben Cuesta-Ramirez, et al.

Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation.General mixed and costly optimization problems are therefore of a great practical interest, yet their resolution remains in a large part an open scientific question. In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables. The continuous space is more easily harvested by classical Bayesian optimization techniques than a mixed space would. Discrete variables are recovered either subsequently to the continuous optimization, or simultaneously with an additional continuous-discrete compatibility constraint that is handled with augmented Lagrangians. Several possible implementations of such Bayesian mixed optimizers are compared. In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space. Among the algorithms involving latent variables and an augmented Lagrangian, a particular attention is devoted to the Lagrange multipliers for which a local and a global estimation techniques are studied. The comparisons are based on the repeated optimization of three analytical functions and a beam design problem.


page 15

page 16

page 18

page 19


Bayesian optimization of variable-size design space problems

Within the framework of complex system design, it is often necessary to ...

Mold into a Graph: Efficient Bayesian Optimization over Mixed-Spaces

Real-world optimization problems are generally not just black-box proble...

Bayesian task embedding for few-shot Bayesian optimization

We describe a method for Bayesian optimization by which one may incorpor...

When to be Discrete: Analyzing Algorithm Performance on Discretized Continuous Problems

The domain of an optimization problem is seen as one of its most importa...

Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian

An augmented Lagrangian (AL) can convert a constrained optimization prob...

Causal Clustering for 1-Factor Measurement Models on Data with Various Types

The tetrad constraint is a condition of which the satisfaction signals a...

Efficient Bayesian Structural Equation Modeling in Stan

Structural equation models comprise a large class of popular statistical...