An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization

10/27/2022
by   Elvin Lo, et al.
0

Molecule optimization is an important problem in chemical discovery and has been approached using many techniques, including generative modeling, reinforcement learning, genetic algorithms, and much more. Recent work has also applied zeroth-order (ZO) optimization, a subset of gradient-free optimization that solves problems similarly to gradient-based methods, for optimizing latent vector representations from an autoencoder. In this paper, we study the effectiveness of various ZO optimization methods for optimizing molecular objectives, which are characterized by variable smoothness, infrequent optima, and other challenges. We provide insights on the robustness of various ZO optimizers in this setting, show the advantages of ZO sign-based gradient descent (ZO-signGD), discuss how ZO optimization can be used practically in realistic discovery tasks, and demonstrate the potential effectiveness of ZO optimization methods on widely used benchmark tasks from the Guacamol suite. Code is available at: https://github.com/IBM/QMO-bench.

READ FULL TEXT
research
02/19/2023

Optimization Methods in Deep Learning: A Comprehensive Overview

In recent years, deep learning has achieved remarkable success in variou...
research
12/23/2020

Gradient-Free Adversarial Attacks for Bayesian Neural Networks

The existence of adversarial examples underscores the importance of unde...
research
05/15/2023

Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks

This work examines the challenges of training neural networks using vect...
research
07/23/2021

Implicit Rate-Constrained Optimization of Non-decomposable Objectives

We consider a popular family of constrained optimization problems arisin...
research
06/22/2022

Sample Efficiency Matters: A Benchmark for Practical Molecular Optimization

Molecular optimization is a fundamental goal in the chemical sciences an...
research
04/16/2022

Analytical Benchmark Problems for Multifidelity Optimization Methods

The paper presents a collection of analytical benchmark problems specifi...
research
02/01/2022

Tutorial on amortized optimization for learning to optimize over continuous domains

Optimization is a ubiquitous modeling tool that is often deployed in set...

Please sign up or login with your details

Forgot password? Click here to reset