GACEM: Generalized Autoregressive Cross Entropy Method for Multi-Modal Black Box Constraint Satisfaction

02/17/2020
by   Kourosh Hakhamaneshi, et al.
9

In this work we present a new method of black-box optimization and constraint satisfaction. Existing algorithms that have attempted to solve this problem are unable to consider multiple modes, and are not able to adapt to changes in environment dynamics. To address these issues, we developed a modified Cross-Entropy Method (CEM) that uses a masked auto-regressive neural network for modeling uniform distributions over the solution space. We train the model using maximum entropy policy gradient methods from Reinforcement Learning. Our algorithm is able to express complicated solution spaces, thus allowing it to track a variety of different solution regions. We empirically compare our algorithm with variations of CEM, including one with a Gaussian prior with fixed variance, and demonstrate better performance in terms of: number of diverse solutions, better mode discovery in multi-modal problems, and better sample efficiency in certain cases.

READ FULL TEXT
research
10/15/2020

Safe Model-based Reinforcement Learning with Robust Cross-Entropy Method

This paper studies the safe reinforcement learning (RL) problem without ...
research
05/25/2018

Learning Self-Imitating Diverse Policies

Deep reinforcement learning algorithms, including policy gradient method...
research
06/24/2022

Black Box Optimization Using QUBO and the Cross Entropy Method

Black box optimization (BBO) can be used to optimize functions whose ana...
research
10/16/2018

Real-Valued Evolutionary Multi-Modal Optimization driven by Hill-Valley Clustering

Model-based evolutionary algorithms (EAs) adapt an underlying search mod...
research
10/25/2021

Goal-Aware Cross-Entropy for Multi-Target Reinforcement Learning

Learning in a multi-target environment without prior knowledge about the...
research
05/03/2023

Black-box Optimizers vs Taste Shocks

We evaluate and extend the solution methods for models with binary and m...
research
10/02/2017

Unsupervised Learning for Nonlinear PieceWise Smooth Hybrid Systems

This paper introduces a novel system identification and tracking method ...

Please sign up or login with your details

Forgot password? Click here to reset