Monte Carlo Elites: Quality-Diversity Selection as a Multi-Armed Bandit Problem

04/18/2021
by   Konstantinos Sfikas, et al.
11

A core challenge of evolutionary search is the need to balance between exploration of the search space and exploitation of highly fit regions. Quality-diversity search has explicitly walked this tightrope between a population's diversity and its quality. This paper extends a popular quality-diversity search algorithm, MAP-Elites, by treating the selection of parents as a multi-armed bandit problem. Using variations of the upper-confidence bound to select parents from under-explored but potentially rewarding areas of the search space can accelerate the discovery of new regions as well as improve its archive's total quality. The paper tests an indirect measure of quality for parent selection: the survival rate of a parent's offspring. Results show that maintaining a balance between exploration and exploitation leads to the most diverse and high-quality set of solutions in three different testbeds.

READ FULL TEXT

page 6

page 7

03/19/2022

Thompson Sampling on Asymmetric α-Stable Bandits

In algorithm optimization in reinforcement learning, how to deal with th...
04/25/2012

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems

Multi-armed bandit problems are the most basic examples of sequential de...
10/23/2019

Diversifying Database Activity Monitoring with Bandits

Database activity monitoring (DAM) systems are commonly used by organiza...
03/15/2016

Optimal Sensing via Multi-armed Bandit Relaxations in Mixed Observability Domains

Sequential decision making under uncertainty is studied in a mixed obser...
07/19/2012

The Road to VEGAS: Guiding the Search over Neutral Networks

VEGAS (Varying Evolvability-Guided Adaptive Search) is a new methodology...
02/19/2019

Bayesian Exploration with Heterogeneous Agents

It is common in recommendation systems that users both consume and produ...
05/23/2017

A Multi-Armed Bandit to Smartly Select a Training Set from Big Medical Data

With the availability of big medical image data, the selection of an ade...

Code Repositories

Monte-Carlo-Elites

Quality-Diversity Selection as aMulti-Armed Bandit Problem


view repo