Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization

02/17/2022
by   Brandon Trabucco, et al.
0

Black-box model-based optimization (MBO) problems, where the goal is to find a design input that maximizes an unknown objective function, are ubiquitous in a wide range of domains, such as the design of proteins, DNA sequences, aircraft, and robots. Solving model-based optimization problems typically requires actively querying the unknown objective function on design proposals, which means physically building the candidate molecule, aircraft, or robot, testing it, and storing the result. This process can be expensive and time consuming, and one might instead prefer to optimize for the best design using only the data one already has. This setting – called offline MBO – poses substantial and different algorithmic challenges than more commonly studied online techniques. A number of recent works have demonstrated success with offline MBO for high-dimensional optimization problems using high-capacity deep neural networks. However, the lack of standardized benchmarks in this emerging field is making progress difficult to track. To address this, we present Design-Bench, a benchmark for offline MBO with a unified evaluation protocol and reference implementations of recent methods. Our benchmark includes a suite of diverse and realistic tasks derived from real-world optimization problems in biology, materials science, and robotics that present distinct challenges for offline MBO. Our benchmark and reference implementations are released at github.com/rail-berkeley/design-bench and github.com/rail-berkeley/design-baselines.

READ FULL TEXT
research
07/14/2021

Conservative Objective Models for Effective Offline Model-Based Optimization

Computational design problems arise in a number of settings, from synthe...
research
12/31/2019

Model Inversion Networks for Model-Based Optimization

In this work, we aim to solve data-driven optimization problems, where t...
research
09/15/2022

Bidirectional Learning for Offline Infinite-width Model-based Optimization

In offline model-based optimization, we strive to maximize a black-box o...
research
03/31/2023

Conflict-Averse Gradient Optimization of Ensembles for Effective Offline Model-Based Optimization

Data-driven offline model-based optimization (MBO) is an established pra...
research
02/16/2021

Offline Model-Based Optimization via Normalized Maximum Likelihood Estimation

In this work we consider data-driven optimization problems where one mus...
research
01/07/2023

Bidirectional Learning for Offline Model-based Biological Sequence Design

Offline model-based optimization aims to maximize a black-box objective ...
research
09/20/2023

Parallel-mentoring for Offline Model-based Optimization

We study offline model-based optimization to maximize a black-box object...

Please sign up or login with your details

Forgot password? Click here to reset